Compute, Storage & Data Services

Identity Services

Root User of an AWS Account

Root User Privileges

• The root user of an AWS account (essentially the account owner, not an IAM identity) should not be used for daily tasks, because it cannot be limited by permissions
  • Create an administrative IAM user for regular admin activities
  • Protect the root user with MFA and securely store its access keys
• The root user is still required for certain key actions, and these may appear on exams
• Primary privileges of the root user to remember:
  • Modify account settings (account name, email, root password, access keys)
  • Close or delete the AWS account
  • Change or cancel the AWS Support Plan
  • Register as a seller in the Reserved Instance Marketplace
    • This allows you to sell unused reserved instance capacity if you purchased a multi-year reservation but don’t need the full term
• Additional/root-only tasks (good to know, not mandatory to memorize):
  • Some billing-related controls
  • Viewing certain tax invoices
  • Restoring IAM user permissions
  • Enabling MFA on S3 buckets
  • Editing or removing S3 bucket policies with invalid VPC IDs or endpoints
  • Signing up for AWS GovCloud

AWS IAM (Identity and Access Management) – CLF-C02

IAM Identities

1. Users (long-term credentials)
   • Typically represents an individual who can log in to the AWS account.
2. Groups (containers for users)
   • Simplifies management: assigning a policy to a group automatically applies it to all its members.
   • Note: IAM groups cannot log in to the AWS account. Only users and roles can.
3. Roles (temporary credentials)
   • Commonly used by AWS services (e.g., EC2, Lambda) to perform actions on resources on your behalf; the service assumes the role and the credentials automatically expire.
   • Roles can also be used to provide federated access for external users (e.g., a user logging in via Facebook can assume a role to access AWS resources).

IAM Policies

• Policy: JSON document that specifies permissions (allow or deny access to AWS services and resources).
• Identity-based policies: Attached to IAM identities (users, groups, or roles).
• Resource-based policies: Directly attached to a resource to control which identities can access it.
  • Example: An S3 bucket policy allowing access only from specific IP ranges.

IAM Security Best Practices

• Use root user only for initial account setup (e.g., creating admin users) or closing the account. Avoid using it for everyday tasks.
• Enforce strong passwords for IAM users.
• Enable MFA (Multi-Factor Authentication) for added account security.
• Apply the principle of least privilege: give only the permissions necessary to complete tasks.
• Protect your credentials: never share IAM users or access keys.

IAM Audit Tools

• IAM Credential Report (account-level)
  • Lists all users in the account and the status of their credentials.
  • Exported as CSV; useful to identify users who haven’t rotated passwords or keys recently.
• IAM Access Advisor (user-level)
  • Shows which services a user has permissions for and when they last accessed them.
  • Useful for reviewing and adjusting user permissions.

Shared Responsibility Model for IAM

• AWS responsibilities:
  • Maintaining global infrastructure security
  • Performing configuration and vulnerability assessments
  • Ensuring compliance certifications are met
• Customer responsibilities:
  • Managing and monitoring IAM users, groups, roles, and policies
  • Enforcing MFA for all accounts
  • Rotating access keys regularly
  • Using IAM tools to assign appropriate permissions
  • Reviewing access patterns and adjusting permissions as needed

Other Identity Services

Advanced Identity Services

• AWS STS (Security Token Service): provides temporary credentials with limited permissions
  • Used whenever an entity assumes an IAM role (sts:assumeRole)
    • Roles can grant access within the same account or across accounts
    • Supports identity federation, allowing external identities (Google, Facebook, etc.) to assume roles and access AWS resources
    • Service roles allow AWS services to act on your behalf (e.g., a Lambda execution role permits a function to write logs to CloudWatch)
• Amazon Cognito: user management for web and mobile apps
  • IAM accounts have a hard limit of 5000 users, so for apps with millions of users, Cognito is the scalable alternative
  • Supports login via federated identities from social providers such as Google and Facebook
• AWS Directory Service: connect an existing user directory to AWS
  • Integrates with Microsoft Active Directory (AD), commonly used for Windows servers and centralized management of users, computers, printers, and file shares
  • Supports other directories, such as SAMBA, or proxies for on-premises directories
• IAM Identity Center: single sign-on for multiple AWS accounts and applications
  • Formerly known as AWS Single Sign-On (AWS SSO)
  • Supports SAML 2.0, EC2 Windows logins, and cloud business apps (e.g., Salesforce, Microsoft 365)
  • Can use a built-in identity store or connect to third-party stores like AD, OneLogin, or Okta
  • Enables centralized identity management across multiple accounts via AWS Organizations

Compute Services

Amazon EC2 (Elastic Compute Cloud) – CLF-C02

EC2 Instance – Key Concepts

• Instance = Virtual Machine (VM) = Virtual Server (VS)
  • Runs on a physical EC2 host
  • Deployed inside a VPC subnet in a single Availability Zone (AZ)
  • Provides full OS-level access
• Configurations:
  • Operating System: Linux, Windows, MacOS
  • Instance type: general purpose, compute-optimized, memory-optimized, storage-optimized, GPU/accelerated
  • Size: CPU + RAM
  • Storage: local instance storage (temporary) or network-attached EBS (Elastic Block Store)
  • User Data: bootstrap script that runs once when the instance launches
  • Instance Role: IAM role granting temporary permissions for the instance to access other AWS resources
  • ENI (Elastic Network Interface): manages instance network IPs
    • Security enforced via VPC Security Groups (firewall rules)

SSH Protocol

• SSH allows secure command-line access to a remote Linux host (Shell CLI)
  • Runs on TCP port 22
• SSH keys enable login to EC2 instances for management
  • Linux and MacOS can connect natively
  • Older Windows versions may require PuTTY
• Instance Connect: browser-based SSH connection
  • No need to download keys
  • Works automatically on Amazon Linux 2

EC2 Purchasing Options

1. Shared Host (default): instances share hardware with other customers
   • On-Demand: pay-per-second, no discounts, ideal for short or variable workloads
   • Spot Instances: use spare EC2 capacity at high discounts, interruptible workloads only
     • Not suitable for critical web servers or databases
   • Reserved Instances: commit 1–3 years for discounted rates, for steady workloads
     • Convertible RIs allow changes to instance type, family, OS, or tenancy
   • Capacity Reservations: reserve capacity in a specific AZ or region
     • Guarantees availability, but does not reduce costs
     • Zonal = specific AZ, higher priority; Regional = flexible AZ, lower priority
2. Dedicated Instances: hardware is shared among your instances only
   • Provides extra security isolation
3. Dedicated Host: full control of a physical EC2 host
   • Billed for the host, not individual instances
   • Useful for server-bound licenses tied to sockets or cores

Savings Plans (1–3 years)

• Commit to a specific amount of compute usage for a discount
• Usage beyond the plan is billed at on-demand rates
• Plans can cover EC2 only or multiple compute services (EC2, ECS, Lambda)

Shared Responsibility Model for EC2

• AWS responsibilities:
  • Physical infrastructure and network security
  • Isolating instances on shared hardware
  • Replacing faulty hardware
  • Compliance management
• Customer responsibilities:
  • OS patching and software updates
  • Installing software and utilities inside the instance
  • Configuring Security Groups
  • Assigning IAM roles to instances and managing IAM users
  • Securing data stored on the instance

EC2 Resilience & Scaling: ELB & ASG

Infrastructure – Key Concepts

• Resilience: a system’s ability to recover automatically from failures
  • AZ-resilient: data replicated within an Availability Zone; can survive individual hardware failures
  • Regionally-resilient: data replicated across a region; can survive an entire AZ failure
• High Availability (HA): ensures maximum uptime by recovering quickly from failures
  • May still have brief downtime, but far shorter than a standard system
  • Related to resilience: resilience = ability to self-heal, HA = ability to self-heal quickly
• Fault Tolerance (FT): system continues running without any downtime despite component failures
  • Typically requires redundant hardware
  • FT is more robust than HA
• Scalability: ability to adjust resources to match demand
  • Overprovisioning: too much capacity → wasted costs
  • Underprovisioning: too little capacity → poor performance
  • Goal: match resources to load automatically
  • Vertical scaling (Scale UP/DOWN): increase/decrease server size
    • Simple, but costly and limited
  • Horizontal scaling (Scale OUT/IN): add/remove identical servers
    • Ideal for distributed systems, requires stateless servers and auto-scaling setup
    • Supports HA and resilience
• Elasticity: automatic scaling of resources based on load, optimizing cost and performance
• Agility: ability to provision or terminate resources very quickly
  • Supports experimentation
  • Not the same as scalability
• Right-sizing: selecting instance types and sizes that fit your workload efficiently
  • Avoid overprovisioning; leverage cloud elasticity
  • Important before migration and continuously after onboarding

Elastic Load Balancing (ELB)

• Load Balancer (LB): distributes traffic across multiple backend EC2 instances
  • Provides a single DNS endpoint instead of one per instance
  • Can span multiple AZs → improves HA

• Health checks: only route traffic to healthy instances
• LB Types:
  1. Classic (CLB): legacy, not recommended
  2. Application (ALB): Layer 7, HTTP/HTTPS traffic
  3. Network (NLB): Layer 4, TCP/UDP, very high performance
  4. Gateway (GWLB): Layer 3, for network security traffic (GENEVE protocol)

EC2 Auto Scaling Groups (ASGs)

• Purpose: provide horizontal scaling and resilience for EC2 workloads
  • Automatically launches or terminates instances based on system load
  • Replaces unhealthy instances automatically
• Configuration parameters:
  • MIN_CAPACITY, DESIRED_CAPACITY, MAX_CAPACITY
  • ASG maintains the number of instances at DESIRED_CAPACITY
  • Scaling policies: adjust DESIRED_CAPACITY automatically
    1. Manual Scaling
    2. Dynamic Scaling: Simple, Step, Target Tracking, Scheduled
    3. Predictive Scaling
• Integration with ELB: set the ASG’s instances as targets in the ELB target group for load distribution

Other Compute Services

Serverless Compute

• Serverless: run applications or code without managing servers
  • AWS handles infrastructure setup, execution, and teardown automatically
  • Reduces administrative overhead for developers
• Developers focus on writing functions, not configuring servers
• Pay-per-use model: you pay only for resources used during execution
  • Ideal for intermittent or unpredictable workloads
• Many AWS services are serverless or offer serverless modes

Typical Serverless Architecture

• AWS Lambda: Function-as-a-Service (FaaS), short-lived, on-demand, scalable code execution
• Amazon API Gateway (APIGW): expose APIs (REST or WebSocket) to invoke Lambda functions
• Event-Driven Architecture (EDA): trigger actions in response to events (e.g., S3 uploads, CRON schedules)
  • Common services: Lambda, APIGW, S3, EventBridge, DynamoDB
• Examples:

Scheduled daily jobs (CRON)

Serverless thumbnail generation

Containerized Compute

• Containers: package an application with its runtime environment
  • Portable, lightweight, predictable behavior on any machine/OS
  • Easier to scale than traditional VMs
• Docker: popular container technology
  • Can run many containers on an EC2 instance

AWS Container Services

• ECS / Fargate / EKS / Batch lectures on Udemy
• Amazon ECS (Elastic Container Service): runs Docker containers
  • Can use EC2 instances or Fargate (serverless containers)
• AWS Fargate: serverless container execution; no EC2 management
• Amazon ECR (Elastic Container Registry): store Docker images with versioning
  • Docker Hub equivalent in AWS
• Amazon EKS (Elastic Kubernetes Service): deploy Kubernetes clusters
  • K8s orchestrates containers on EC2 or Fargate
• AWS Batch: execute batch jobs from Docker images
  • Launches resources only during job execution, then shuts them down

Amazon Lightsail

• Lightsail: simplified cloud platform for launching virtual servers, storage, DBs, and networking
  • Designed for beginners or simple workloads
• Pros:
  • Easy to use
  • Predictable, low pricing
• Cons:
  • Limited scalability (some HA available)
  • Fewer AWS integrations compared to EC2/RDS

Storage Services

Amazon S3 (Simple Storage Service) – CLF-C02

Amazon S3 – Security

• User-based access: controlled via IAM policies (allow/deny actions on S3)
• Resource-based access:
  • Bucket Policies: define which principals can access the bucket and which actions are allowed/denied
    • Supports cross-account and public (external) access
    • IAM Access Analyzer: helps optimize bucket policies and analyze access patterns
  • Block Public Access: ON by default; overrides other settings to prevent public access
  • Access Control Lists (ACLs): legacy, simple object/bucket-level permissions; avoid if possible
• Encryption: protect S3 data from unauthorized access
  • In-transit: enforce HTTPS (SSL/TLS)
  • At-rest: client-side (CSE) or server-side (SSE) encryption

S3 Static Website Hosting

• Can host static websites on S3
  • Configure a root document (e.g., index.html)
  • AWS provides a default URL based on bucket name
• Content must be static (S3 objects only)
• Security considerations:
  • Disable Block Public Access to make the website accessible externally
  • Bucket policy must allow public read; otherwise, 403 Forbidden

Additional S3 Features

• Versioning:
  • Keeps multiple versions of an object; updating does not overwrite
  • Disabled by default
  • Deleting creates a deletion marker, not actual deletion → prevents accidental loss
• Replication:
  • Asynchronous replication from one bucket to another (same-region or cross-region)
  • Requires versioning enabled
• Storage Classes: trade-offs between cost and access speed
  1. Standard – default
  2. Infrequent Access (IA) – cheaper storage, pay for access
  3. One Zone-IA (1Z-IA) – cheaper, replicated in single AZ
  4. Glacier – archival, very low cost, longer retrieval times
     • Instant, Flexible, Deep Archive
  • Intelligent Tiering: automatically moves objects based on access patterns
  • Lifecycle Policies: move objects to another storage class on a schedule, not usage

Shared Responsibility Model – S3

• AWS responsibility:
  • Global infrastructure security, durability, availability
  • Can sustain simultaneous loss of data in two facilities
  • Unlimited storage, encryption support, data separation, no access by AWS employees
  • Configuration, vulnerability analysis, compliance validation
• Customer responsibility:
  • Bucket policies & public access settings
  • Data encryption (at rest & in transit)
  • Logging & monitoring access
  • Versioning & replication setup
  • Choosing S3 storage classes

Storage for Private Services (e.g. EC2)

Storing EC2 Data

• EC2 instance data is stored in either:
  • Instance Store: internal block storage in the EC2 host (ephemeral)
  • Network-based storage: persistent external storage
    • Block storage: EBS volumes
    • File storage: EFS or FSx file systems
• Every EC2 instance requires an attached boot/root volume
  • Usually an EBS volume
  • Contains data required for the OS to boot
  • Additional storage volumes can be attached

EC2 Instance Store

• High-performance hardware disk hosted on the same EC2 host
  • Very fast: high IOPS, low latency
• Ephemeral storage: data is lost if the instance stops or is terminated
• Not all EC2 instance types support instance store volumes

Amazon Elastic Block Store (EBS)

• Persistent block storage accessible over the network
  • Like an external USB drive for your instance
• Characteristics:
  • Tied to a single AZ
  • Instances can have 0+ EBS volumes (boot volume is typical)
  • Usually attached to one instance at a time
  • Slower than instance store due to network latency
  • Persistence:
    • Boot volumes: usually deleted with instance termination
    • Other volumes: persist by default after instance termination
• EBS Snapshots: backups of volumes stored in S3
  • Can restore new volumes from snapshots
  • Supports cross-AZ and cross-region restores

EC2 Amazon Machine Image (AMI)

• AMI = static EC2 template with pre-configured OS, software, and data
• Can launch multiple instances from the same AMI (cloning)
• AMI Marketplace: vendors sell pre-built AMIs
• EC2 Image Builder: automates building, testing, and distributing AMIs

Amazon Elastic File System (EFS)

• Network file system for Linux EC2 instances (NFS protocol)
• Can be attached to multiple instances in a region
  • Shared file system across Linux instances
• More expensive than EBS
• EFS-Infrequent Access (EFS-IA): cost-optimized storage for infrequently accessed files

Amazon FSx

• Provides networked file storage for specific use cases
• Main FSx options:
  • FSx for Windows: file system for Windows servers
  • FSx for Lustre: high-performance Linux file system for HPC, big data, or ML workloads

Shared Responsibility Model – Private Storage

• AWS responsibility:
  • Infrastructure
  • Data replication for EBS, EFS, FSx
  • Replace faulty hardware
  • Ensure AWS employees cannot access customer data
• Customer responsibility:
  • Backup/snapshot procedures
  • Data encryption
  • Managing data on volumes/drives
  • Understanding risks of ephemeral storage (Instance Store)

Database & Data Services

Databases 101

• Databases help organize stored data into a defined format, making it easier to search, retrieve, and analyze
  • You can establish schemas, indexes, and relationships to structure the data
  • Similar to turning unorganized notes into a well-structured document with sections, references, and navigation aids, which improves accessibility

Relational Databases = SQL Databases = RDBMS

• Use structured tables with predefined schemas, consisting of rows (records) and columns (fields)
  • Queries are performed using SQL (Structured Query Language)
• Tables behave similarly to spreadsheets, where data can be linked across multiple tables

Relational database table structure (rows and columns with relationships)

• Examples
  • Open source: MySQL, PostgreSQL, MariaDB
  • Commercial: Oracle SQL, Microsoft SQL Server
• Two primary optimization approaches
  • Row-oriented (OLTP – Online Transaction Processing)
    • Designed for frequent, real-time transactions
    • Examples: Amazon RDS, Amazon Aurora
  • Column-oriented (OLAP – Online Analytical Processing)
    • Designed for large-scale analytics and reporting
    • Example: Amazon Redshift

Non-relational Databases = NoSQL Databases

• Provide flexible data models without strict schemas
  • Do not rely on standard SQL (though some support SQL-like query languages)
  • Typically offer better scalability but may trade off some consistency
• Examples
  • Document databases (e.g., MongoDB)
    • Store data in formats like JSON

{
  "name": "Avatar",
  "year": 2009,
  "genre": "epic science fiction",
  "director": {
    "name": "James Cameron",
    "nationality": "Canada"
  }
}

• Graph databases (e.g., Amazon Neptune)
• Key-value databases (e.g., Amazon DynamoDB)

Example of a key-value table with primary key and attributes

Databases in AWS

• You can install and run database software on virtual machines such as Amazon EC2
  • However, this requires managing the infrastructure yourself, which increases administrative effort
• Preferred approach: AWS managed database services
  • Faster deployment and easier scaling
  • Built-in high availability and backup capabilities
  • Automatic updates and patching
  • Integration with monitoring and other AWS tools
  • Less direct control compared to self-managed solutions

AWS SQL Database Services

• Amazon RDS (Relational Database Service)
  • Row-based (OLTP), included in free tier options
  • Runs inside your VPC with your chosen database engine
  • Features
    • Read Replicas for distributing read workloads
    • Multi-AZ deployments for failover and high availability
• Amazon Aurora
  • AWS-developed relational database engine
  • Higher performance than standard RDS engines, typically at higher cost
  • Offers a serverless option for automatic scaling
• Amazon Redshift
  • Column-based (OLAP), built for analytics workloads
  • Operates within a VPC using clusters
  • Serverless mode available
• AWS Database Migration Service
  • Enables secure database migration with minimal downtime
  • Supports migrations across different database engines
  • Primarily designed for SQL-based migrations, with limited NoSQL support (e.g., DynamoDB as a target)

Shared Responsibility Model for RDS

• AWS responsibilities
  • Manage underlying infrastructure and restrict OS-level access
  • Apply database and OS patches
  • Maintain hardware reliability
• Customer responsibilities
  • Configure network access (ports, IPs, security groups)
  • Manage database users and permissions
  • Control public accessibility
  • Configure encryption (in transit via SSL and at rest)

AWS NoSQL Database Services

• Amazon ElastiCache
  • In-memory data store to improve read performance
  • Supports Redis and Memcached
• Amazon DynamoDB
  • Serverless, highly scalable, and low latency
  • Includes DAX (DynamoDB Accelerator) for in-memory caching specific to DynamoDB
• Amazon DocumentDB
  • Managed database compatible with MongoDB
• Amazon Neptune
  • Designed for relationship-heavy datasets
  • Common use cases include recommendation systems, fraud detection, and social platforms
• Amazon Timestream
  • Optimized for time-based data such as metrics, logs, and IoT telemetry
• Amazon QLDB
  • Ledger database with immutable and verifiable transaction records
  • Being phased out (support ended July 31, 2025), but may still appear in exams
• Amazon Managed Blockchain
  • Supports decentralized networks using Hyperledger Fabric and Ethereum
  • Enables transactions without a central authority

AWS Data Engineering & Data Analytics Services

• Amazon EMR
  • Managed big data platform using Hadoop and related tools
• AWS Glue
  • Serverless ETL (Extract, Transform, Load) service
  • Includes a Data Catalog for metadata management
• Amazon Athena
  • Query data directly from S3 using SQL
  • Uses schema-on-read (ELT approach)
  • Charges based on data scanned per query
• Amazon QuickSight
  • Business intelligence tool for dashboards and reporting
  • Integrates with multiple AWS data sources and uses machine learning for insights

Other Storage Services

AWS Snowball

• AWS Snowball
  • Physical devices provided by AWS to gather, process, and transport data outside traditional networks
  • Designed to be secure, rugged, and portable for use in challenging environments
• Use cases
  1. Edge computing
     • “Edge” refers to locations with limited or no reliable internet access
       • Examples include transportation systems, remote industrial sites, or offshore operations
  2. Offline data transfer
     • Used when moving large volumes of data over the network is impractical or too slow
     • Enables data migration into or out of AWS without relying on internet bandwidth

AWS Storage Gateway

• AWS Storage Gateway
  • Provides a hybrid storage solution, combining on-premises systems with cloud storage
  • Acts as a connection layer between local infrastructure and AWS storage services such as Amazon S3
• Common purposes
  • Backup data from on-premises systems to AWS
  • Expand storage capacity by integrating with cloud storage
• Types of Storage Gateway
  1. Volume Gateway → block storage integration between on-prem and AWS
  2. Tape Gateway → replaces traditional backup tapes with virtual tapes stored in AWS
  3. File Gateway → file-based access (e.g., NFS/SMB) backed by Amazon S3