Free ISACA Advanced in AI Audit AAIA Exam Questions

Only reviewing an ML model's performance one time prior to migrating to production (option C) is of greatest concern. The AAIA Study Guide emphasizes that ''AI and ML models require continuous monitoring and periodic performance reviews in production to detect issues such as data drift, model degradation, or evolving risk factors.'' A single pre-production review fails to capture these changes and risks, potentially resulting in undetected failures or compliance issues.

Periodic (including annual) and event-driven reviews are necessary to ensure ongoing model reliability.

ISACA Advanced in AI Audit (AAIA) Study Guide, Section: 'Continuous Monitoring and Review of Deployed AI Models'

Regular threat modeling allows organizations to proactively identify vulnerabilities specific to AI, such as:

Data poisoning

Model inversion attacks

Membership inference attacks

Prompt injection

Unauthorized model manipulation

AAIA highlights AI threat modeling as essential for anticipating and mitigating AI-specific security risks that traditional IT threat modeling does not fully address.

Option C (regulatory compliance) is a secondary benefit. Option D (drift prevention) is unrelated. Option B relates to performance, not security.

The primary purpose is early identification of security weaknesses before exploitation occurs.

AAIA Domain 2: Threats and Vulnerabilities Specific to AI.

AAIA Domain 1: Risk Assessment and Control Planning.

Model degradation occurs as the 'Freshness' of the training data wanes and real-world conditions evolve. The most robust control is 'Periodic human reviews' (Human-in-the-Loop). Human experts can identify 'drift' in logic or common-sense failures that automated systems might miss. Relying on model-generated data (Option A) can lead to 'Model Collapse,' where the AI begins to drift into nonsensical patterns by reinforcing its own previous outputs. Human oversight ensures the model remains grounded in reality and aligned with business objectives.

Bias in AI models is most commonly introduced through the training data. The AAIA Study Guide highlights that to ensure fairness, auditors and developers must evaluate the diversity, representativeness, and quality of the data used to train the model.

''The greatest source of bias in AI comes from the training data. Reviewing and auditing this data is critical to ensuring that outputs do not disproportionately affect specific groups or skew results.''

While adaptability (C) and model parameters like temperature (D) affect behavior, they do not address the root cause of most biases. The development environment (B) supports infrastructure but not ethical assurance.

Incomplete training data often leads to underrepresentation of certain applicant types, products, or scenarios. In credit and lending, this typically translates into systematic bias: some groups are evaluated on richer historical patterns, while others are evaluated on sparse or unrepresentative information. The greatest associated risk is therefore unfair loan decisions (A), which can manifest as unjustified rejections, inappropriate pricing, or inconsistent risk assessments.

While delays (B), reduced satisfaction (C), or increased manual work (D) may occur, they are secondary operational issues. AAIA highlights that for financial services, the central risks include fairness, discrimination, regulatory compliance, and reputational impact. Incomplete data directly undermines fairness and can violate lending regulations and internal risk appetite.

ISACA, AAIA Exam Content Outline -- Domain 1: AI Governance and Risk (risk categories, including fairness and discriminatory outcomes).

ISACA AI ethics content on data completeness and representativeness in decisioning systems.

Representativeness ensures that the training data reflects the full spectrum of conditions the AI model will encounter in production. According to the AAIA Study Guide, models trained on non-representative data are prone to bias, poor generalization, and underperformance in real-world applications.

''Ensuring that training data accurately represents the operational environment is critical for model reliability, fairness, and scalability. Without it, the model may perform well in testing but fail in actual usage.''

Timeliness (A) and understandability (D) support performance and usability, but they are secondary to ensuring data coverage. Predictability (B) may not be desirable in dynamic modeling.

A 'Model Card' is a standardized document that provides essential information about an AI model's intended use, training data, limitations, and performance metrics. For an auditor or risk manager, the primary benefit is gaining a clear 'Understanding of model intent and performance context.' It allows the organization to determine if a vendor's model is fit for the specific business purpose and to identify potential risks (such as data bias or environmental limitations) before acquisition. While it supports documentation compliance (Option A), its core value lies in providing the transparency necessary for informed decision-making and governance.

Ensuring that AI tools are trained on properly licensed and documented data sets is critical to avoiding copyright infringement and legal exposure. The AAIA Study Guide emphasizes using platforms with certified and traceable training data to meet ethical and legal standards.

''Organizations must verify the provenance and licensing of data used to train AI systems. Platforms that certify data sources reduce the risk of using protected intellectual property without consent.''

Manual review (A) is resource-intensive and may not detect embedded copyright violations. Labeling (C) is not sufficient for legal protection. Suspension (D) may be excessive without first attempting remediation. Thus, B is the most strategic and effective recommendation.

The non-deterministic nature of AI refers to the fact that modern AI systems---especially generative AI, reinforcement learning, and probabilistic models---do not always produce the same output even when given identical inputs. According to AAIA's ethical and operational guidance, understanding non-determinism is essential for:

Managing expectations of AI behavior

Preventing overreliance on AI outputs

Identifying hallucinations or inconsistent outcomes

Ensuring human oversight remains in place

Supporting proper audit trails and accountability

Upholding professional skepticism toward AI-generated outputs

If training does not explain non-determinism, employees may mistakenly believe AI outputs are always authoritative or correct, which can lead to unsafe reliance, ethical violations, and audit failures.

Other options (A, C, D) are helpful but not foundational. Lack of understanding of non-determinism fundamentally undermines safe and responsible AI use.

AAIA Domain 5: Ethical and Legal Considerations (Explainability, Output Variability, Responsible Use).

AAIA Domain 3: Oversight and Human-in-the-Loop Principles.

The AAIA Study Guide stresses that inputting confidential or proprietary data into third-party generative AI tools may result in unauthorized data disclosure. These tools may store, process, or retrain on the input data, leading to privacy and intellectual property risks.

''When employees input sensitive data into external AI tools, organizations risk losing control over that information. This may result in regulatory non-compliance, legal exposure, and irreversible data leakage.''

While business disruption (C) and reliance (D) are notable, the most severe and immediate risk is B---unauthorized disclosure. Data contamination (A) impacts model reliability, not data security.

Cross-validation testing is a statistical method used to assess how well a model generalizes to an independent data set. The AAIA Study Guide recommends this method as a best practice to fine-tune model accuracy and reduce both false positives and false negatives. It involves splitting the dataset into training and testing subsets multiple times to ensure model robustness.

''Cross-validation allows auditors and developers to identify overfitting and adjust model parameters to achieve better generalization and predictive accuracy, especially in fraud detection contexts.''

Regression testing (A) focuses on changes over time; substantive testing (C) is audit-specific but not model-focused. Benford's Law (D) applies to numerical distributions but is not designed for optimizing ML models. Hence, B is the best approach.

'Model degradation' (or model decay) happens when a model's performance slowly worsens because the real-world environment has changed since its last training. The most effective safeguard is 'Periodic human reviews.' Humans can identify 'contextual shifts' or 'common-sense errors' that automated monitoring might miss. This 'Human-in-the-Loop' (HITL) control ensures that the AI's decisions remain grounded, justifiable, and accurate over time. Relying on model-generated data (Option A) can actually accelerate degradation through a phenomenon known as 'Model Collapse,' where the AI begins to learn from its own mistakes.

A significantly higher rejection rate is a clear indicator of potential algorithmic discrimination. Management's PRIMARY response should be to conduct a comprehensive bias analysis (C), including fairness metrics, root-cause analysis, model explainability assessments, and data quality reviews. AAIA prioritizes fairness auditing and bias remediation as central to AI governance.

Option A is unacceptable because fairness issues fall outside most risk tolerances. Option B is a procedural check, not the solution. Option D (synthesizing data) might help but only after the root cause is identified---it is not the primary first step.

ISACA, AAIA Exam Content Outline -- Domain 1: Bias, Fairness, and Transparency Evaluations.

Prompt injection attacks involve maliciously crafted inputs intended to override system instructions, exfiltrate data, or cause harmful behavior. The most effective control aligned with incident management is to deploy robust input validation and sanitization (C), which includes rules and filters designed to detect and neutralize potentially malicious content before it reaches the model. AAIA's coverage of AI threats and vulnerabilities highlights the importance of input validation and secure prompt handling for generative AI systems.

Fine-tuning the model (A) is a long-term adaptation, not an immediate incident control. Scanning for code-like structure (B) or excessive special characters (D) may catch some attacks but are too narrow; many prompt injections use natural language. Comprehensive input validation and sanitization is the most effective and generalizable incident management response.

ISACA, AAIA Exam Content Outline -- Domain 5: Ethical and Legal Considerations in AI; AI-specific threats and incident management.

ISACA AI security guidance covering prompt injection and input validation controls.

An AI model inventory documents the models in use, their purposes, and how they support specific business functions. According to the AAIA Study Guide, maintaining a comprehensive AI model inventory allows auditors to trace model objectives, performance metrics, and use cases back to business goals.

''A well-maintained AI model inventory supports governance and alignment by offering a centralized view of model functions, business integration, and ownership. It ensures transparency and strategic coherence.''

While policies and assessments are important, only the inventory directly shows which AI models exist and their connection to organizational objectives.