ISTQB® Certified Tester AI Testing (CT-AI) incl Exam (ISTQB-CT-AI)

Module1: Introduction to AI

• • Definition of AI and AI Effect
• • Narrow, General and Super AI
• • AI-Based and Conventional Systems
• • AI Technologies
• • AI Development Frameworks
• • Hardware for AI-Based Systems
• • AI as a Service (AIaaS)
• • Contracts for AI as a Service
• • AIaaS Examples
• • Pre-Trained Models
• • Introduction to Pre-Trained Models
• • Transfer Learning
• • Risks of using Pre-Trained Models and Transfer Learning
• • Standards, Regulations and AI

Module 2: Quality Characteristics for AI-Based Systems

• • Flexibility and Adaptability
• • Autonomy
• • Evolution
• • Bias
• • Ethics
• • Side Effects and Reward Hacking
• • Transparency, Interpretability and Explainability
• • Safety and AI

Module 3: Machine Learning (ML) – Overview

• • Forms of ML
• • Supervised Learning
• • Unsupervised Learning
• • Reinforcement Learning
• • ML Workflow
• • Selecting a Form of ML
• • Factors Involved in ML Algorithm Selection
• • Overfitting and Underfitting
• • Overfitting
• • Underfitting
• • Hands-On Exercise: Demonstrate Overfitting and Underfitting

Module 4: ML - Data

• • Data Preparation as Part of the ML Workflow
• • Challenges in Data Preparation
• • Hands-On Exercise: Data Preparation for ML, Validation and Test Datasets in the ML Workflow
• • Hands-On Exercise: Identify Training and Test Data and Create an ML Model
• • Dataset Quality Issues
• • Data Quality and its Effect on the ML Model
• • Data Labelling for Supervised Learning
• • Approaches to Data Labelling
• • Mislabeled Data in Datasets

Module 5: ML Functional Performance Metrics

• • Confusion Matrix
• • Additional ML Functional Performance Metrics for Classification, Regression and Clustering
• • Limitations of ML Functional Performance Metrics
• • Selecting ML Functional Performance Metrics
• • Hands-On Exercise: Evaluate the Created ML Model
• • Benchmark Suites for ML

Module 6: ML - Neural Networks and Testing

• • Neural Networks
• • Hands-On Exercise: Implement a Simple Perceptron
• • Coverage Measures for Neural Networks

Module 7: Testing AI-Based Systems Overview

• • Specification of AI-Based Systems
• • Test Levels for AI-Based Systems
• • Input Data Testing
• • ML Model Testing
• • Component Testing
• • Component Integration Testing
• • System Testing
• • Acceptance Testing
• • Test Data for Testing AI-based Systems
• • Testing for Automation Bias in AI-Based Systems
• • Documenting an AI Component
• • Testing for Concept Drift
• • Selecting a Test Approach for an ML System

Module 8: Testing AI-Specific Quality Characteristics

• • Challenges Testing Self-Learning Systems
• • Testing Autonomous AI-Based Systems
• • Testing for Algorithmic, Sample and Inappropriate Bias
• • Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
• • Challenges Testing Complex AI-Based Systems
• • Testing the Transparency, Interpretability and Explainability of AI-Based Systems
• • Hands-On Exercise: Model Explainability
• • Test Oracles for AI-Based Systems
• • Test Objectives and Acceptance Criteria

Module 9:  Methods and Techniques for the Testing of AI-Based Systems

• • Adversarial Attacks and Data Poisoning
• • Adversarial Attacks
• • Data Poisoning
• • Pairwise Testing
• • Hands-On Exercise: Pairwise Testing
• • Back-to-Back Testing
• • A/B Testing
• • Metamorphic Testing (MT)
• • Hands-On Exercise: Metamorphic Testing
• • Experience-Based Testing of AI-Based Systems
• • Hands-On Exercise: Exploratory Testing and Exploratory Data Analysis (EDA)
• • Selecting Test Techniques for AI-Based Systems

Module 10: Test Environments for AI-Based Systems

• • Test Environments for AI-Based Systems
• • Virtual Test Environments for Testing AI-Based Systems

Module 11: Using AI for Testing

• • AI Technologies for Testing
• • Hands-On Exercise:The Use of AI in Testing
• • Using AI to Analyze Reported Defects
• • Using AI for Test Case Generation
• • Using AI for the Optimization of Regression Test Suites
• • Using AI for Defect Prediction
• • Hands-On Exercise: Build a Defect Prediction System
• • Using AI for Testing User Interfaces
• • Using AI to Test Through the Graphical User Interface (GUI)
• • Using AI to Test the GUI.