cert-bank

Guides

이 글을 한국어로 읽기

DBX-GAI Study Guide: Read Every Question as a Design Review With Five Questions

The Databricks Certified Generative AI Engineer Associate exam (DBX-GAI) is less about writing a chatbot than about reviewing someone else's design. A team describes a goal or a broken application, and you decide what it really needs, what is wrong and how to prove the fix. A senior reviewer asks the same five questions every time. This guide teaches the exam through those five questions.

Study guide · Databricks Certified Generative AI Engineer Associate (DBX-GAI)

DBX-GAI Study Guide: Read Every Question as a Design Review With Five Questions

The exam

45 scored questions (plus up to 10 unscored ones) in 90 minutes. Items are multiple choice or multiple selection, so expect some "select two" questions. Databricks does not publish the pass mark. There are six domains: Design Applications (14%), Data Preparation (14%), Application Development (30%), Assembling and Deploying Applications (22%), Governance (8%) and Evaluation and Monitoring (12%).

We wrote 225 DBX-GAI questions for this question bank; 33 of them are "Select TWO". 128 of them (57%) are use-case questions: a goal plus constraints, and you pick the design. The services that come up most are MLflow (63 questions), AI Search (52, under its current and former names) and Foundation Model APIs (33). Expect evaluation and tracing to run through many questions, not only the 12% domain.

Many products have new names. Mosaic AI Vector Search is now Databricks AI Search, AI Gateway is now Unity Gateway, and Genie spaces are now Genie Agents. The exam guide may still use the old names. Learn both, and treat Knowledge Assistant and Supervisor Agent as Legacy, not as the recommended design.

Review question 1: does this even need an agent?

Pick the lightest design that meets the goal. Many wrong options add moving parts that nothing in the stem asks for.

The stem saysLightest fitting design
Thousands of PDFs, fill a table, nobody chatsBatch SQL: ai_parse_document, then ai_extract with a schema, in a scheduled job
One custom prompt for every row of a tableai_query over the table, with responseFormat if code parses the result
Questions over private documents, fixed stepsA simple RAG chain: retriever, prompt template, chat model, output parser
The next step depends on what a lookup returnsAn agent with tools, authored with ResponsesAgent
Business users ask data questions in plain languageA Genie Agent over governed tables, or (in Public Preview on Model Serving) called as a tool by your agent

For private knowledge that changes weekly, retrieval beats tuning: retrieved passages are read at question time, while tuned weights are a snapshot. When a program must parse the reply, use response_format with a JSON schema rather than asking nicely in the prompt.

Review question 2: is the corpus ready to be retrieved?

Data Preparation is about everything that happens before the first query. Choose the parser by format. Text-based PDF and Word files can use a library or ai_parse_document. HTML needs an HTML parser. Scans have no text layer, so they need OCR. Then clean before you chunk. Drop running headers, footers and page numbers (the parser tags them by type), and remove near-duplicates, for example with MinHash. Every repeated chunk competes with a useful one.

Chunking has four families: fixed-size, paragraph-based, format-specific (Markdown or HTML headings) and semantic. The Databricks guidance says fixed-size rarely works for production. Keep modest overlap. Store source, section and page as columns beside each chunk, so retrieval can filter and the answer can cite. The chunks go into a Delta table with change data feed enabled, and a Delta Sync index follows that table.

Beginner tip: do the token arithmetic on paper once, because the exam does it for you in numbers. Take a 4,096-token context window, 500 tokens of instructions and 600 for the answer. Five chunks of 1,500 tokens (7,500) do not fit. Five chunks of 500 tokens give 500 + 2,500 + 600 = 3,600 and do. Then check the embedding side: GTE Large and BGE Large, the two embedding models in the Foundation Model APIs docs that are not in Preview, both output 1,024 dimensions, but GTE Large reads up to 8,192 tokens and BGE Large only 512. A 1,200-token chunk does not fit BGE's 512-token window.

Review question 3: when the answer is wrong, which half failed?

Debug a RAG answer in order. First ask whether retrieval returned the right passages. Then ask whether generation used them. Open the MLflow trace. The retriever span shows what came back, and the model span shows what was written from it.

  • Off-topic passages: a faithful model still writes a fluent wrong answer. Fix retrieval: chunking, metadata filters, hybrid search for exact codes, a reranker, or rewriting a follow-up ("and its battery life?") into a standalone query before retrieval.
  • Right passages, unsupported claims: fix generation with a structured prompt. Put the instructions first, wrap the context in delimiters, and give a fixed reply for "not in the documents".
  • Every reply stops mid-sentence at about the same length: check max_tokens and the finish reason.
Going deeper: why hybrid search rescues product codes

An embedding places text by meaning. A code like "PX-4471" carries almost no meaning, so its vector lands near other codes, and pure similarity search ranks the exact match poorly. Hybrid search in AI Search runs a BM25 keyword search next to the approximate nearest neighbour search (HNSW) and merges the two ranked lists with Reciprocal Rank Fusion. Each document scores by its rank in each list, not by raw scores, which are not comparable between the two methods. A document that is first in the keyword list rises even if its vector is unremarkable. That is why "users search by SKU or error code" points to query_type="hybrid", not to a bigger embedding model.

Review question 4: who may call what?

Permission questions come in two directions. Who may call the endpoint is set by its ACL. An application that only sends requests needs CAN QUERY. CAN MANAGE also lets someone change and delete the endpoint. What the deployed chain may reach is set by the resources you declare when logging: the AI Search index, the Unity Catalog functions, the chat endpoint, a Genie Agent. At deployment, Databricks checks that the endpoint creator can use each resource, creates a service principal for the model version and injects short-lived credentials.

Registration has its own rules. Use a three-level Unity Catalog name. Creating a model needs USE CATALOG, USE SCHEMA and CREATE MODEL, and a new version needs ownership or CREATE MODEL VERSION. Aliases, not stages, mark which version serves. Files the model needs at serving time are packaged with artifacts. Settings that differ per environment, such as the endpoint name or the prompt text, go in ModelConfig.

Going deeper: why "it works in my notebook" fails on the endpoint

In a notebook, your agent runs as you, with your grants. A serving endpoint runs in its own container under its own identity. It can reach only what was provisioned for it, and Databricks provisions credentials only for the resources declared at logging time. An index that the code calls but never declared gets no credential, so the call fails with a permission error even though the code is unchanged. The fix is to declare the resource, make sure the endpoint creator holds the right permission on it (for example CAN USE on an index or CAN RUN on a Genie Agent), and redeploy.

Review question 5: how do you know it works?

Evaluation runs before release, on a fixed dataset, with mlflow.genai.evaluate(). Monitoring runs the same scorers on sampled live traces after release (MLflow production monitoring is in Beta). To compare two versions, keep the dataset and the scorers identical and change only the app. The exam often hinges on which judges need ground truth:

Needs expectations (ground truth)Needs only inputs, outputs and the trace
Correctness, RetrievalSufficiency, ToolCallCorrectnessRelevanceToQuery, RetrievalRelevance, RetrievalGroundedness, Safety, Guidelines, ToolCallEfficiency

A questions-only dataset or live traffic can run only the right-hand column. For cost, separate the features that limit usage (rate limits, which return HTTP 429, and Unity Gateway budgets, which alert or block) from those that only show it (inference tables, traces, the system.ai_gateway.usage table).

The five questions a reviewer asks of every gen AI design
1
Does it need an agent?
Pick the lightest design that meets the goal: batch SQL, ai_query, a RAG chain, then an agent
2
Is the corpus ready?
Parser by format, clean, then chunk with source, section and page beside each chunk
3
Which half failed?
Open the MLflow trace: retrieval first, then generation
4
Who may call what?
Endpoint ACL for callers; resources declared at logging for what the chain reaches
5
How do you know it works?
Same dataset and scorers before release; monitoring on live traces after
Most DBX-GAI stems test one of these five questions; ask them in this order when reviewing a design.

A study order

  1. Application Development first (30%): prompts, chunking by evaluation, model selection, guardrails, ResponsesAgent.
  2. Assembling and Deploying (22%): pyfunc packaging, resources, Unity Catalog registration, AI Search index types and endpoints, ai_query batch work.
  3. Design and Data Preparation together, since they are one path from goal to index.
  4. Evaluation and Monitoring, then Governance (masking at ingestion, injection defences, licences). Then mixed sets.

Practise

Practise in the question bank: every question explains each option, right and wrong. Start with the domain you just studied, then take a timed mock exam.

Domains and weights as listed in the Databricks exam guide for the Generative AI Engineer Associate certification. Several features named here are renamed or still in Preview or Beta; check the current exam guide and documentation.