ISTQB Certified Tester AI Testing Practice Test

These questions are part of the practice quiz. Start practicing

• What is a model card used for in AI testing and governance?
• What is drift detection and what algorithms are commonly used in AI monitoring?
• What are common sources of bias in AI?
• Why is calibrating probability outputs important in AI decision systems like loan approvals?
• How does reinforcement learning differ from supervised learning?
• What is the role of test datasets in the ML lifecycle?
• Which statement best describes autonomy in AI-based systems?
• Which practices help ensure reproducibility in AI experiments?
• In testing autonomous AI systems, what is a key objective?
• How is A/B testing applied to AI-based systems?
• Which characteristics make AI-based systems challenging to use in safety-related applications?
• What testing challenge is exacerbated by the complexity and opacity of AI systems?
• What are the typical test items included in an AI test plan?
• AI in UI testing improves robustness by which means?
• In regression test optimization, AI can.
• Which principle is part of ethical AI governance?
• Which practice helps prevent data leakage?
• How does test data governance impact AI testing outcomes?
• Which dataset quality issue can lead to biased model predictions due to imbalanced classes?
• What testing challenge is associated with self-learning AI systems?
• What does a reliability diagram demonstrate in calibration assessment?
• How can experience-based testing be applied to AI-based systems?
• Why are test oracles difficult for AI-based systems?
• How should exploratory testing be conducted on an AI-infused application?
• Which statement about ML performance metrics across classification, regression, and clustering is true?
• Which metamorphic relations are commonly used in image classification testing?
• What is a straightforward capability of AI in test case generation?
• Which components are essential for monitoring AI model inference in production to ensure reliability?
• Data preparation aims to produce which kind of dataset?
• Which practice helps ensure that a model's predictions remain valid when data distributions shift?
• What kind of monitoring is important for AI production systems?
• In the ML workflow, which step is performed after understanding objectives and before preparing and testing data?
• How does Explainable AI (XAI) relate to testing AI systems and what should testers assess?
• Which statement about AI as a Service (AIaaS) is true?
• How should a failure mode and effects analysis be adapted for AI systems?
• Which option lists the data labeling approaches that can be used for supervised learning?
• What constitutes data leakage in AI model training?
• Which technologies are used to implement AI?
• Describe the structure of a neural network including a DNN.
• What is the purpose of deploying a trained ML model?
• What is a metamorphic relation in AI testing, and what is a typical example?
• In unsupervised learning, what do clustering and association do?
• Why are flexibility and adaptability important for AI-based systems?
• What are the three core pillars of AI testing according to ISTQB AI Testing, and how do they interact?
• What is drift detection used for in production AI systems?
• How can fairness be measured in AI systems, and what testing approach supports this?
• How does reinforcement learning work?
• Which elements are included in a typical foundation-level AI test plan?
• Which statement best describes biased data and its impact on AI fairness and performance?
• What is data drift and why is it important in AI testing?
• Which aspects are tracked by data lineage in AI systems?
• Why does probabilistic and non-deterministic behavior complicate testing AI systems?
• Which hardware option is typically best for small-scale ML workloads?
• Which statement best describes a typical AI-assisted defect analysis workflow?
• Which testing approach helps prevent adversarial attacks and data poisoning in ML systems?
• Why is data quality assessment critical in AI testing, and which key dimensions should be evaluated?
• How should test data be constructed to evaluate model generalization?
• What is the role of ML benchmark suites?
• Which is a challenge in data preparation?
• What role do data preprocessing and feature engineering play in AI testing?
• In testing user interfaces, AI primarily enhances robustness through which capability?
• Why is data lineage important for traceability, reproducibility, and compliance?
• What should be included in deployment readiness validation for an AI model?
• Which of the following describes a risk of unmanaged AI evolution?
• What is the difference between system-level testing and acceptance testing for AI systems?
• Because AI outputs can be non-deterministic, what is often required to draw meaningful conclusions?
• Which of the following is a technique used to test robustness against adversarial inputs?
• What is the bias-variance tradeoff and its impact on AI testing strategies?
• Which statement correctly contrasts overfitting and underfitting?
• Why is monitoring latency and throughput important in AI inference services, and how should it be tested?
• Which statement defines robustness testing in AI and provides a typical example?
• How can AI technologies be categorized in software testing?
• What is typically measured by ML benchmark suites when comparing AI technologies?
• What is concept shift in AI, in terms of the input-output relationship?
• Which statement best describes General AI?
• What best describes a test environment for AI model inference in production?
• Which statement best captures the differences between AI testing and traditional software testing?
• Which of the following is a popular AI development framework?
• Which model evaluation strategy helps mitigate data leakage during testing?
• Which statement best describes Narrow AI, General AI, and Super AI?
• What defines metamorphic testing in relation to inputs and outputs?
• Which of the following describes a failure mode and its test in AI systems?
• What describes metamorphic testing in AI-based systems?
• Which AI technique is used for test case generation and optimization in software testing?
• What is the purpose of using cross-domain data in generalization testing?
• What does the AI Effect describe?
• Why is managing evolution of AI systems important?
• What is back-to-back testing in AI context?
• What is concept drift and how does it differ from data drift in AI systems?
• In risk-based AI testing, which aspects are typically prioritized?
• What is a key limitation of relying solely on ML functional performance metrics to judge ML system quality?
• Data preparation in ML typically involves which activities?
• What is the difference between unit testing and integration testing in AI components?
• Which factors does AI use for defect prediction?
• What activities are involved in testing for bias in AI systems?
• Concept drift affects what in AI systems?
• Which ethical principle directly supports user trust by making model decisions understandable?
• What is a typical outcome when metamorphic testing is applied to AI systems?
• Which factor makes testing AI-based systems difficult related to data management?
• Why is it challenging to create test oracles for AI-based systems?
• Which data quality dimension refers to how up-to-date the data is?
• What is a surrogate model in AI testing and how is it used as a test oracle?
• What are the benefits provided by virtual test environments for AI-based systems?
• Which statement best describes how data quality can influence model bias?
• Which description accurately reflects the reasons data in datasets may be mislabeled?
• Comprehensive documentation of an AI component supports testing by providing transparency into what aspects?
• Why include sandboxed data in an isolated production-like test environment for AI models?
• Which statement is correct about training, validation, and test datasets in ML model development?
• Which statement best captures the relationship between autonomy duration and human intervention for AI systems?
• In AI-assisted UI testing, which approach reduces fragility caused by UI changes?
• How can you test the reliability of AI systems in production?
• What is the purpose of a test traceability matrix in AI testing?
• Which statement explains testing AI-based systems in relation to system specifications?
• Which statement best describes deterministic AI systems?
• Which items would be included in a fairness-focused testing checklist for an AI-powered loan system?
• How is pairwise testing used for AI-based systems?
• Which elements should be included in a scenario-based fairness and compliance checklist for an AI-powered loan approval system?
• How can concept shift be detected in AI pipelines?
• What is a key difference between AI-based systems and conventional rule-based systems?
• Which statement about the test oracle problem in AI-based test case generation is true?
• Explain the concept of test oracles for AI and list two practical oracle approaches.
• What data quality checks should be performed before training an AI model?
• What is the purpose of a confidence interval in AI test results?
• Name two test techniques commonly used in AI testing.
• How do robustness checks under perturbations help validate AI systems?
• Which aspect helps determine whether a problem is framed as supervised or unsupervised learning?
• What is the recommended approach to versioning data and models in AI testing?
• Which activities follow training in the ML workflow?
• Which statement about transparency in AI-based systems is true?
• Which ML form is typically used when there are labeled data and the output is a category?
• Which of the following best describes how a test traceability matrix supports AI testing?
• What distinguishes conventional systems from AI-based systems?
• Which of the following describes neural network coverage measures?
• How can transparency and explainability be tested in AI systems?
• Describe testing for multi-model ensembles and potential pitfalls?
• What is automation bias and how can it affect AI testing?
• What is the primary benefit of AI-optimized regression suites?
• Which privacy-preserving techniques should testers verify in AI testing, and what should be checked?
• Why is frequent testing of trained models necessary?
• What is a test oracle problem unique to AI, and how can metamorphic testing help?
• What best describes the role of governance and ethics in AI testing?
• Which statement best describes the impact of poor data quality on machine learning models?
• What does online deployment monitoring entail?
• Which evaluation metrics are commonly used for binary classification in AI testing and what do they measure?
• Which statement correctly describes data labeling approaches for supervised learning?
• In AI-assisted test case generation, which inputs can AI leverage to create test assets?
• To reflect real-world changes, AI testing needs to include what type of input data?
• What are replay tests in AI testing and when are they used?
• How can AI optimize regression test suites?
• AI-assisted defect prediction relies on which data sources?
• Which components are typically included in a model card?
• Using the same data acquisition and pre-processing methods as data scientists can introduce what risk?
• Which concept is highlighted as part of Google's ML test checklist within experience-based testing?
• Which describes how AI can support the analysis of new defects?
• Which practices support reproducibility by ensuring experiments can be repeated with the same data and results?
• What is the primary consequence of bias in AI systems?
• How are AI-based systems tested at different test levels?
• What is a risk of using pre-trained AI models without modification?
• What is data leakage in AI testing and how can it be prevented?
• What is offline hyperparameter tuning vs online experimentation in AI testing?
• Which hardware is ideal for edge computing in AI?
• Which of the following best describes transparency, interpretability, and explainability in AI?
• What are main factors differentiating test environments for AI-based systems from conventional systems?
• Which method best assesses calibration in probabilistic AI models?
• Which practice describes the use of simulations of big data inputs in AI testing environments?
• Which of the following is a widely used AI framework?
• What is the purpose of backpropagation in training a neural network?
• Which statement best reflects the AI-assisted test case generation and the test oracle problem?
• How do standards apply to AI-based systems?
• In supervised learning, what is classification and what is regression?
• In AI-enabled UI testing, the application can be interacted with at which level?
• Which is a typical dataset quality issue?
• What are side effects and reward hacking in AI?
• What is the role of monitoring in AI deployment testing?
• Feature engineering is the process of transforming raw data into a dataset suitable for training by
• What data-related requirement often constrains AI testing?
• How does the ISTQB AI Testing framework apply risk-based testing?