Question 1
CASE STUDY
Please use the following to answer the next question:
You have recently assumed the role of AI Governance leader for a California-based medical technology company. The organization primarily serves hospitals and has recently expanded to include walk-in clinics located within local pharmacies.
The company's core business focuses on diagnostic assistance powered by a large language model LLM and back-office process optimization using Agentic AI, including chatbots, medical record request handling, scheduling and billing.
In preparation for its next round of funding, the board has asked you to prepare an AI Risk report to demonstrate to investors how the company is addressing AI-related risks. In preparing the report you learn that last year the company generated 30 million dollars in gross revenue across the US, EU, India, and South Korea and that vendors are engaged for various activities, including model testing and providing third-party AI solutions for chatbots.
Which of the following best exemplifies human oversight capabilities you should enable under the relevant AI laws?
The correct answer is A because it directly demonstrates meaningful human oversight over AI-generated outcomes, which is a key requirement in AI governance frameworks and regulations such as the EU AI Act. Human oversight requires that a qualified human can review, intervene, and override AI decisions before they produce legal or significant real-world effects. In high-risk contexts like healthcare diagnostics, governance frameworks emphasize ''human-in-the-loop'' controls to prevent harm and ensure accountability. Option A ensures a licensed medical professional validates the AI output before it is finalized, aligning with safety, accountability, and risk mitigation principles. Other options describe training, system design, or monitoring, which are important governance measures but do not constitute direct oversight of individual AI decisions at the point of impact, making them insufficient under strict regulatory expectations.