Fundamentals of AI and ML

Security, Compliance and Governance  

Conventional IT Infrastructure

Client-Server Paradigm for Web Applications

• A client application (for example, your web browser) sends a request over the internet (a network of networks) to a remote server (such as a Gmail server).
• The server continuously listens for incoming requests from clients. When a request arrives, it can accept it and establish a connection with the client.
• Clients and servers recognize each other using their IP addresses.

Physical Servers and Hardware (HW)

Computer/Server Components

1. COMPUTE: responsible for processing data and performing calculations
   • Hardware components: CPU (Central Processing Unit), GPU (Graphics Processing Unit)
   • GPUs are faster and more expensive than CPUs → ideal for high-performance tasks such as gaming graphics, big data analysis, machine learning training, and Generative AI.
2. MEMORY: temporarily holds frequently or recently accessed data for fast access
   • Data in memory is temporary/ephemeral, meaning it can be lost or overwritten (e.g., when opening new programs or restarting the server)
   • Hardware components: RAM (Random Access Memory)
3. STORAGE: persistently saves data, though access is slower than memory
   • Hardware components: SSD (Solid State Drive), HDD (Hard Disk Drive)
   • Data can be organized in various structures: block storage, object storage, file storage (file systems)
   • Databases (DBs): store data with additional structure and intelligence, making searching (querying) and processing more efficient than standard storage
4. NETWORK (NW): handles communication with external devices, sending and receiving data
   • Hardware components: cables, Network Interface Card (NIC), L3 Router, L2 Switch, DNS server, etc.
   • Networking often involves multiple intermediate devices to ensure proper routing across a network

• Physical hardware can be emulated using technologies like virtualization and containerization, allowing virtual components and servers to use physical resources more efficiently

Challenges of Physically Hosted Servers

• Traditionally, IT organizations hosted servers on their premises or in Data Centers (DCs)

• Example: Google started in a garage with its own servers before expanding

Common challenges:

• Unpredictable scaling: difficult to forecast user numbers or usage patterns (time of day, week, or year) → risk of under- or over-provisioning
• Wasted capacity: buying large servers as a precaution often leaves unused resources in the early stages
• Management overhead: engineers must update software, monitor servers 24/7, and handle hardware upgrades/replacements
• Maintenance costs: servers require space, secure facilities, power, and cooling
• Low resilience: physically localized servers are vulnerable to natural disasters, vandalism, or other incidents
• High startup costs: purchasing and setting up physical infrastructure is expensive
• Cloud computing addresses many of these issues, allowing customers to rent infrastructure and avoid the challenges of managing physical servers

Cloud Computing

• Definition: A model that provides ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., servers, networks, storage, applications, services).
  • These resources can be rapidly allocated and released with minimal management effort or provider interaction.
• Cloud computing has transformed how businesses and individuals access and utilize computing resources.
  • Benefits: increased scalability, greater flexibility, reduced maintenance, less physical infrastructure

Five Key Characteristics of a Cloud Platform

• A cloud platform is defined by five main characteristics:
  • On-Demand Self-Service
  • Broad Network Access
  • Resource Pooling
  • Rapid Elasticity
  • Measured Service

On-Demand Self-Service

• Users can provision resources as needed without requiring human intervention.
  • Management can be done via UI, CLI, or APIs.

Broad Network Access

• Resources are available over the network and accessed through standard mechanisms.
  • No specialized devices or non-standard network protocols are required.

Resource Pooling

• Resources are shared among multiple users using a multi-tenant model.
• Users experience location independence → they do not know or control the exact location of resources.
• Economies of scale: buying hardware in bulk reduces costs, allowing cloud providers to offer more affordable services.

Rapid Elasticity

• Resources can be quickly scaled up or down based on demand.
  • Avoid overprovisioning → prevents wasted expenses
  • Avoid underprovisioning → maintains performance
• To users, resources appear unlimited.
• This is one of the most important advantages of cloud computing.

Measured Service

• Resource usage can be monitored, controlled, reported, and billed.
• Pay only for what you consume (pay-as-you-go).
  • Important: Always terminate resources you no longer need to avoid unexpected costs.

Six Advantages of Cloud Computing (AWS Whitepaper)

1. Shift from CAPEX to OPEX
   • Convert fixed capital expenses into variable operational expenses
   • Reduces Total Cost of Ownership (TCO) and operational costs
   • Customers pay on demand, without owning hardware
2. Benefit from massive economies of scale
   • Large-scale cloud providers like AWS can offer lower prices due to efficiency
3. Eliminate capacity guessing
   • Scale resources based on actual usage, avoiding idle infrastructure
4. Increase speed and agility
   • Provision or release resources faster than owning physical hardware
   • Enables cost-effective experimentation and development
5. Reduce data center management costs
   • Less time and money spent maintaining infrastructure, allowing focus on business and customers
6. Deploy globally in minutes
   • Launch applications in multiple regions with minimal effort
   • Utilize global cloud infrastructure for lower latency and improved user experience

Public vs Private vs Multi vs Hybrid Cloud

Types of Cloud Computing

Public Cloud

• A cloud computing platform that is openly available to the general public
  • Users can access, provision, and manage resources over the public internet
• Leading public cloud service providers include:
  • Amazon Web Services (AWS)
  • Microsoft Azure
  • Google Cloud Platform (GCP)

Overview Diagram of Cloud Computing Models

• The use of multiple public cloud platforms to deploy and operate applications
• Common implementation examples include:
  • Hosting different components of an application architecture across separate cloud providers
  • Mirrored cloud architecture, where the complete application stack is deployed across two providers (e.g., AWS and Azure)
    • If one cloud environment becomes unavailable, the application remains accessible through the other, ensuring high availability (HA)
• Recommendation: Avoid single-pane-of-glass management interfaces for multi-cloud environments
  • These tools abstract vendor-specific details
    • This makes it difficult to identify which capabilities and advantages belong to each provider
  • Only shared features across vendors are typically exposed
    • Provider-exclusive services and innovations may not be visible

Private Cloud

• An on-premises cloud platform dedicated to a single organization
• Major public cloud providers also offer private, on-premises cloud solutions, such as:
  • AWS Outposts
  • Azure Stack
  • Google Anthos
• To qualify as a true private cloud, the platform must still meet the five essential characteristics of cloud computing
  • Traditional on-premises virtualization platforms (e.g., VMware, Hyper-V, XenServer) are not equivalent to a private cloud
    • Although they may offer cloud-like features, they do not satisfy all five required cloud computing characteristics

Hybrid Cloud

• A model that combines private cloud and public cloud resources, operating together as a single, integrated environment
  • The same tools, processes, and management interfaces are used across both public and private cloud components
• Using public cloud services alongside on-premises infrastructure alone constitutes a hybrid environment or hybrid networking, not a hybrid cloud
  • To be classified as a hybrid cloud, both public cloud and private cloud platforms must be fully integrated

Overview Diagram of Cloud Computing Models

Cloud Service Models (Everything as a Service – XaaS)

Infrastructure Stack (App Stack)

• The infrastructure stack represents the layers involved in deploying a software application:
  1. Application
  2. Data – data stores, databases, etc.
  3. Runtime Environment (RTE) – code libraries and dependencies
  4. Container – Docker, Kubernetes (K8s), etc.
  5. Operating System (OS) – Linux, Windows, macOS, etc.
  6. Virtualization
  7. Server – physical machine or computer
  8. Infrastructure – server racks, network routers, and related hardware
  9. Facilities – building, electricity, real estate, security personnel, etc.
• Unit of consumption: The component of the stack that the customer pays for and consumes
  • All layers below the unit of consumption are managed by the vendor
  • The unit of consumption itself and all layers above are managed by the customer
  • Different cloud service models define different units of consumption
• On-Premises Deployment
  • Customers are responsible for all layers of the infrastructure stack
  • High cost, significant administrative overhead, and operational risks
  • Maximum control and flexibility over the environment
• Data Center (DC) Hosted Deployment
  • Customers pay primarily for facility usage (fully or partially) while managing other layers themselves
  • Common practice prior to the widespread adoption of cloud computing

Popular Cloud Service Models

Infrastructure-as-a-Service (IaaS)

• Unit of consumption: Operating System (OS)
• IaaS provides virtualized servers or machines (VMs)
• Pricing: Pay-as-you-go
  • Charges are based on seconds, minutes, or hours of VM usage
  • No costs are incurred when VMs are not in use
• Offers reduced cost and operational risk, with some trade-offs in flexibility
• One of the most widely adopted cloud services
• Example: Amazon Elastic Compute Cloud (EC2)

Platform-as-a-Service (PaaS)

• Unit of consumption: Runtime Environment (RTE)
• Designed for developers who want to deploy and run applications without managing underlying infrastructure
• Example platforms: Heroku, AWS Elastic Beanstalk

Software-as-a-Service (SaaS)

• Unit of consumption: Application
• Users access the application directly without exposure to other infrastructure layers
  • Limited control over tools and customization
  • Minimal costs and operational risks
• Pricing: Usually subscription-based (monthly or yearly), sometimes with free tiers or trial options
• Examples: Netflix, Dropbox, Gmail

Other Cloud Services

• Function-as-a-Service (FaaS): e.g., AWS Lambda
• Database-as-a-Service (DBaaS): e.g., MongoDB Atlas
• Container-as-a-Service (CaaS): e.g., Amazon Elastic Container Service (ECS)
• …and more specialized offerings

Overview Diagram