Section 26: AI Governance
Why AI Governance Matters
Effective AI governance helps organizations:
- Protect individuals and sensitive information
- Minimize bias and prevent data exposure
- Improve reliability and consistency
- Reduce unexpected system behavior
- Support transparency and explainability
- Maintain clear and auditable records
Structuring AI Governance
AI Center of Excellence (CoE)
An AI CoE may oversee the organization’s AI initiatives by:
- Establishing standards
- Sharing best practices
- Coordinating activities across departments
Hub-and-Spoke Model
This structure balances:
- Innovation: Business units independently experiment with AI
- Governance: A centralized team maintains standards and oversight
AI Handoffs
Successful AI operations require coordination among:
- Product owners
- Engineering and data teams
- Security, legal, and risk professionals
- Executive leadership
Organizations typically progress through maturity stages, evolving from informal AI experimentation to fully governed and optimized AI programs.
AI Policies and Procedures
Components of an AI Policy Framework
| Layer | Required? | Purpose |
|---|---|---|
| Policies | Yes | Define high-level principles and rules |
| Standards | Yes | Specify mandatory technical or operational requirements |
| Procedures | Yes | Provide step-by-step compliance processes |
| Guidelines | No | Recommend best practices |
Important AI Governance Questions
Organizations should clearly define:
- Who is authorized to use AI?
- Who approves AI-related projects?
- How are AI initiatives governed?
- Which decisions require human involvement?
- What information must be communicated to stakeholders?
- What are the data labeling and access requirements?
- Which controls apply to vendors and systems?
- What information must be logged and retained?
- Which data sources are permitted for training?
- Who requires AI-related training?
- How should future AI investment and improvement be managed?
AI governance frameworks should remain flexible and evolve alongside technology and regulations.
AI-Related Roles
| Area | Roles |
|---|---|
| Data Modeling | Data Scientists, Data Engineers, ML Engineers |
| Architecture & Platforms | AI Architects, Platform Engineers, MLOps Engineers |
| Security & Governance | AI Security Architects, Governance Engineers, Risk Analysts, AI Auditors |
Organizations should establish clear responsibilities and separation of duties based on operational needs.
Section 27: AI Risks
Responsible AI Principles
| Principle | Meaning |
|---|---|
| Fairness | Prevent discriminatory outcomes |
| Reliability & Safety | Ensure systems operate safely and predictably |
| Transparency | Clearly communicate system behavior and limitations |
| Privacy | Protect personal and sensitive information |
| Security | Preserve confidentiality, integrity, and availability |
| Differential Privacy | Add controlled statistical noise to protect identities |
| Explainability | Provide understandable reasons for decisions |
| Inclusiveness | Ensure accessibility for diverse users |
| Accountability | Ensure responsibility for AI outcomes |
| Consistency | Maintain stable behavior across situations |
Categories of AI Risk
AI risk is commonly evaluated using:
Risk = Impact × Exposure × Uncertainty
| Risk Type | Description |
|---|---|
| Bias Introduction | Reinforcing societal inequalities or inaccurate assumptions |
| Accidental Data Leakage | Exposing confidential information unintentionally |
| Reputational Damage | Loss of public trust after AI failures |
| Poor Accuracy or Performance | Incorrect or delayed outputs |
| Intellectual Property Risks | Copyright or ownership violations |
| Autonomous Misbehavior | AI acting beyond intended limits |
Organizations should balance innovation and risk management rather than completely restricting AI adoption.
Bias Introduction
Bias occurs when AI systems reflect social or historical inequalities rather than objective patterns.
Bias Mitigation Approaches
- Clear purpose definition
- Data profiling
- Labeling standards
- Balanced sampling
- Privacy-preserving synthetic data
Prevention Methods
- Fairness metrics
- Class balancing
- Adversarial debiasing
- Post-processing adjustments
- Red-team testing
Bias management requires continuous monitoring because risks evolve over time.
Accidental Data Leakage
Data leakage often results from collecting unnecessary or overly sensitive information.
Mitigation Strategies
- Data minimization
- Sensitive data labeling
- Encryption
- Differential privacy
- Secrets management
- Redaction controls
- Guardrails
- Rate limiting
Organizations should also establish dedicated incident response plans for AI-related data exposure.
Reputational Damage
Overstating AI capabilities can create trust failures when systems underperform.
Best Practices
- Communicate capabilities honestly
- Monitor public perception
- Publish transparency or trust reports
Early reputational monitoring helps identify problems before they escalate.
Accuracy and Performance
Accuracy
Measures how often outputs are correct.
Performance
Measures speed and responsiveness, including:
- Latency
- Throughput
Organizations should establish measurable benchmarks before deployment.
Canary Releases
A small percentage of live traffic is routed to a new model version before full deployment to limit operational risk.
Ongoing monitoring helps identify both performance degradation and model drift.
Intellectual Property (IP) Risks
Potential Issues
- Training data may contain copyrighted material
- AI outputs may reproduce protected content
- Ownership of generated content may be unclear
Mitigation Strategies
- Maintain strong provenance tracking
- Define clear policies for data sourcing and output usage
- Monitor outputs for IP violations
- Keep detailed documentation records
Autonomous Systems
AI autonomy exists on a spectrum ranging from fully human-controlled to fully autonomous.
Key Governance Questions
- What happens if the system fails?
- How are unexpected situations handled?
- What fallback mechanisms exist?
Monitoring and governance establish boundaries for autonomous actions and verify safe operation.
Shadow IT and Shadow AI
Shadow IT
Use of unauthorized hardware, software, or cloud services.
Shadow AI
Use of unapproved AI systems, such as employees submitting confidential information into public AI chatbots.
Recommended Approach
Instead of banning AI entirely, organizations should provide approved and secure AI alternatives to reduce unsanctioned use.
Awareness Training
Employees should receive training on responsible AI use and organizational expectations.
Target Audiences
- All employees
- Technical teams
- Legal, audit, and compliance professionals
Using realistic examples and storytelling improves training effectiveness.
Section 28: AI Compliance
EU AI Act
The EU AI Act classifies AI systems according to risk levels.
| Risk Tier | Regulatory Treatment | Examples |
|---|---|---|
| Prohibited Practices | Completely banned | Government social scoring |
| High-Risk Systems | Strict assessments and oversight required | Hiring systems, law enforcement AI |
| Limited-Risk Systems | Transparency obligations apply | AI chatbots |
| Minimal-Risk Systems | Few mandatory requirements | Spam filters, game AI |
| General Purpose AI (GPAI) | Broad transparency obligations | Large language models |
Violations may result in penalties reaching up to 7% of global annual revenue.
OECD AI Principles
The OECD introduced one of the first international AI governance frameworks.
Core Principles
- Human rights protection
- Inclusiveness and sustainability
- Transparency
- Security and robustness
- International cooperation
The framework also encourages investment in research and workforce development.
ISO AI Standards
| Standard | Purpose |
|---|---|
| ISO 22989 | Defines AI terminology and concepts |
| ISO 23053 | Describes ML system frameworks and workflows |
| ISO 23894 | Provides AI risk management guidance |
NIST AI Risk Management Framework (AI RMF)
| Function | Purpose |
|---|---|
| GOVERN | Establish accountability and governance structures |
| MAP | Identify and understand AI risks |
| MEASURE | Assess risks through testing and evaluation |
| MANAGE | Implement controls and continuous improvements |
Example
A healthcare organization might:
- Govern ownership and responsibilities
- Map patient safety concerns
- Measure model performance across demographics
- Manage identified risks before deployment
Corporate AI Policies
Approved vs. Unapproved AI Tools
| Sanctioned Tools | Unsanctioned Tools |
|---|---|
| Approved by IT, Legal, and Security | Not formally reviewed |
| Risks are managed | Risks are unknown |
| Example: Internal AI assistant | Example: Public chatbot used with company data |
Public vs. Private AI Models
| Public Models | Private Models |
|---|---|
| Hosted by third-party providers | Hosted internally or in private environments |
| Higher data exposure risk | Greater organizational control |
| Best for non-sensitive tasks | Best for regulated or confidential workloads |
Organizations should define clear rules regarding what information may be used with public AI platforms.
Third-Party Compliance Assessments
Independent reviewers verify that AI solutions align with applicable regulations, standards, and organizational policies. The evaluation process generally involves:
- Preparation Assessment — determine how existing controls and practices compare to compliance expectations.
- Evidence Review — examine supporting records, system logs, policies, and configuration settings.
- Validation Testing — conduct technical and operational evaluations of the system.
- Fairness and Effectiveness Analysis — measure accuracy, reliability, and potential bias against established benchmarks.
- Reporting and Recommendations — provide a comprehensive assessment report outlining observations, gaps, and suggested improvement
Data Sovereignty
Data sovereignty establishes rules based on the geographic location where data is stored, managed, and processed, rather than solely on data ownership.
Primary Requirements:
- Data Residency Mandates: Information must remain within designated national or regional boundaries.
- Local Processing Obligations: Data handling and processing activities must occur on infrastructure located within the required jurisdiction.
- Cross-Border Data Restrictions: The movement of data between countries is governed by strict regulatory controls (for example, Chapter V of the GDPR in the European Union).
AI technologies introduce additional sovereignty challenges because model training and inference operations may unintentionally route data through cloud environments spanning multiple jurisdictions.
Risk Reduction Measures: Implement data classification policies, data governance controls, and regional data management strategies to maintain compliance with sovereignty requirements