Free Databricks Certified Context Engineer Associate Context-Engineer-Associate Exam Questions

MLflow 3 provides an integrated observability, experiment tracking, prompt management, and evaluation environment for generative AI applications developed on Databricks.

The application requires end-to-end visibility into inference operations, including prompts, retrieved passages, model responses, token usage, latency, and feedback.

MLflow Tracing records individual execution spans associated with model invocations, retrievers, tools, and other agent components. These traces enable engineers to reconstruct the execution path and investigate failures or performance bottlenecks.

MLflow's Prompt Registry provides centralized prompt-template versioning, while experiment tracking associates application configurations with their observed performance.

The evaluation framework supports automated scoring, human feedback, and comparisons between application versions.

Option A describes governance capabilities associated with Unity Catalog but does not independently provide the complete MLflow evaluation workflow. Option B would require implementing application-specific logging and analysis functionality. Option D concerns tool interoperability rather than experiment observability.

Option C is therefore the appropriate selection.

One technical distinction is that MLflow manages prompt versions through its Prompt Registry, not simply by storing them as conventional model artifacts. The registry can track lineage between prompts and logged application or model versions.

Together, these capabilities support reproducible experimentation, operational monitoring, and performance analysis without requiring a custom tracking backend.

Official Databricks reference: Tracing overview, Prompt Registry, Evaluate and improve, and Track prompt versions alongside application versions.

The application incorrectly combines short-term conversational memory and long-term semantic memory within a single vector retrieval mechanism.

Short-term memory must preserve the sequence of messages, recent user statements, intermediate decisions, and tool interactions within an active session.

Semantic vector retrieval does not guarantee chronological completeness. It retrieves information based on relevance scores, potentially omitting earlier conversation details that remain essential for the current interaction.

Furthermore, storing all users' conversation summaries in a shared index without appropriate access boundaries can cause unrelated information to appear in generated responses.

Option B separates these responsibilities.

A session-scoped short-term buffer maintains the active conversation context, preserving continuity across turns. A separate persistent vector store retains durable information such as confirmed preferences, customer-specific facts, and significant decisions that should remain accessible across sessions.

Databricks provides this architectural distinction through Agent Sessions and Agent Memory, both of which can use Lakebase-backed persistence.

Option A improves retrieval isolation but continues using semantic search as the primary mechanism for session continuity. Option C does not properly distinguish durable memory from session history. Option D continues accumulating redundant summaries and increases retrieval overhead.

Production implementations should also enforce user-scoped access controls, bounded session histories, and relevance-based long-term memory retrieval.

Official Databricks reference: Agent memory and sessions, Agent sessions, and Agent state and memory.

The failure results from treating session-level context compaction as a substitute for persistent, cross-session task memory.

Compaction reduces the amount of information maintained within an active context window. However, a compacted conversation does not automatically become available to subsequent, independently initialized sessions.

Option C addresses this limitation by introducing a durable external memory mechanism for preserving task progress.

At the end of each research session, the agent should persist structured records containing completed competitor analyses, unresolved research questions, important findings, source references, and the next required actions.

When the next session begins, the agent retrieves these records and reconstructs the relevant working state without replaying the entire conversation.

Databricks provides Lakebase-backed Agent Memory for persisting information across conversations. Application-managed storage can also retain structured workflow checkpoints associated with a particular research task.

Option A affects compaction timing but does not create durable persistence. Option B increases context consumption and eventually violates the stated context-window constraint. Option D stores extensive transcripts without efficiently distinguishing completed work from incidental conversation.

For reliable long-horizon execution, memory records should include task identifiers, completion status, timestamps, and references to supporting evidence.

This design enables continuation across sessions while maintaining predictable context consumption and minimizing redundant research.

Official Databricks reference: Agent memory and sessions, Agent memory, and Develop agents on Databricks.

The evaluation indicates that the compaction process is losing important information despite producing summaries comfortably within the available context budget.

This is primarily a context-retention failure rather than a token-efficiency problem.

Option C correctly prioritizes improving information recall before attempting further compression.

The engineer should revise the compaction instructions to explicitly preserve previous resolutions, escalation decisions, unresolved issues, ownership changes, customer commitments, and significant troubleshooting events.

These details may remain important even when they occurred many turns earlier.

For example, an escalation decision made at turn 8 may determine how a support case should be handled at turn 40. A summarizer that prioritizes recent messages may incorrectly omit that earlier decision.

Option A would make summaries shorter and potentially worsen information loss. Option B assumes recent events are always more important. Option D changes both retention and pruning behavior simultaneously, making it harder to identify which modification improves performance.

A disciplined optimization process should first establish adequate coverage of important facts, then remove redundant language while preserving that coverage.

Databricks MLflow evaluation supports testing application behavior against representative datasets and comparing quality across configurations.

The engineer should use a benchmark containing expected escalation events and resolutions to assess completeness before optimizing summary length.

Official Databricks reference: Tracing overview, MLflow API Reference, and Debug and analyze your app with tracing.

The observed failure results from a mismatch between semantic relevance and regulatory applicability.

Semantic similarity measures how closely document content relates to a query. It does not independently establish whether a regulatory document applies to a specific jurisdiction, reporting period, or effective date.

Consequently, a highly relevant US regulation may rank above an applicable UK regulation when both discuss similar financial reporting obligations.

Option B resolves this problem through metadata-constrained retrieval.

Databricks AI Search supports query filters that restrict results according to indexed metadata fields. The engineer can filter documents by jurisdiction and applicable regulatory status before incorporating their contents into the agent's context.

Effective-date filtering should account for the requested reporting date and the organization's regulation-versioning rules. A document marked active today is not necessarily the regulation that governed an earlier reporting period.

After applying the eligibility constraints, semantic retrieval can identify the most relevant passages from the remaining documents.

Option A increases candidate volume without correcting applicability. Option C sacrifices valuable semantic matching. Option D relies on generation-time instructions to compensate for incorrect retrieval.

The engineer should also enforce access permissions independently; metadata filters improve retrieval correctness but are not a substitute for authorization.

Official Databricks reference: AI Search filtering guide and Improve RAG chain quality --- Metadata Filtering.

Option C provides governance enforcement at the service boundary rather than depending on probabilistic agent behavior.

Databricks supports registering external MCP servers as Unity Catalog securable objects. This allows administrators to control which principals may access the service and which tools are exposed.

For this scenario, the engineer should grant the appropriate service permissions to authorized compliance officers and restrict the exposed tool set to approved operations.

Databricks service policies can inspect MCP requests before execution through the ON CALL evaluation phase.

A policy can return an ASK decision, requiring user approval before the requested operation proceeds.

The engineer should configure this behavior specifically for submit_regulatory_report, while permitting authorized read-only operations without unnecessary approval prompts.

The destructive delete_compliance_record operation should be excluded or denied according to the organization's authorization requirements.

Option A depends entirely on instruction following and cannot reliably enforce privileged operations. Option B changes reasoning behavior without adding authorization controls. Option D unnecessarily removes the legitimate report-submission capability.

Databricks also supports auditing MCP activity, enabling administrators to review tool usage and investigate unauthorized or unexpected actions.

Service policies are currently documented as a Beta capability, so the relevant preview must be enabled before this mechanism can be used.

Official Databricks reference: Service policies for AI securables, Connect agents to third-party tools with MCP Services, and AI governance guide.