cert-bank

Guides

이 글을 한국어로 읽기

DBX-MLP Study Guide: Study the Hand-offs, Because That Is Where Production ML Breaks

The Databricks Certified Machine Learning Professional exam (DBX-MLP) assumes you can already train a model. What it tests is everything around the training: how features reach the model without leaking the future, how a model moves from one environment to the next, how it gets served, and how you notice when it goes bad. Most failures in those areas happen at a hand-off, where one stage passes something to the next. This guide is built around those hand-offs.

Study guide · Databricks Certified Machine Learning Professional (DBX-MLP)

DBX-MLP Study Guide: Study the Hand-offs, Because That Is Where Production ML Breaks

The exam and where the points are

59 scored multiple-choice questions in 120 minutes, plus up to 10 unscored items. Databricks does not publish the pass mark, so check the official exam page. There are three domains: Model Development (44%), MLOps (44%) and Model Deployment (12%). The two big domains count equally, so do not spend most of your time on modelling and leave MLOps for the end.

We wrote 295 DBX-MLP questions for this question bank. 104 of them (35%) are troubleshooting scenarios: something is slow, leaks, drifts or returns a 401, and you pick the cause or the fix. Each of our questions has exactly one correct answer. The services that come up most are Unity Catalog, Model Serving, MLflow and data profiling (the feature formerly called Lakehouse Monitoring). Each appears in more than 50 questions.

Hand-off 1: from feature tables to the training set

The classic MLP trap is a feature table that a nightly job overwrites. It keeps only today's row for each customer. Join twelve months of labels to it and every old label gets today's feature values. The model learns from the future, offline metrics look excellent, and production quality collapses.

The fix is a time series feature table. Its primary key includes a timestamp column declared TIMESERIES. A FeatureLookup then names a timestamp_lookup_key from the label DataFrame. For each label, the join returns the latest feature row at or before that timestamp. This has nothing to do with Delta time travel, which reads one snapshot of the whole table.

Two follow-up rules come up often. First, log the model with the Feature Engineering client (fe.log_model(..., training_set=training_set)). Then batch scoring and Model Serving repeat the same lookups, and callers send only keys. A plain mlflow.pyfunc.log_model records no feature dependencies. Second, a value that exists only in the request, such as the amount of the current transaction, cannot be precomputed. It becomes an on-demand feature: a Python UDF registered in Unity Catalog and bound with FeatureFunction, so training and serving run the same code.

Beginner tip: make the leak happen once on purpose. Build a tiny label table with three dates and a feature table that you overwrite twice. Join them the naive way and look at which feature values each old label gets. Then declare the date as a TIMESERIES key and build the training set again. Seeing the rows change is what makes point-in-time questions easy.

Hand-off 2: from one machine to many

Scaling questions are a ladder. Climb it in this order and stop at the first step that fits:

  1. Does it fit on one node? Train there, on a bigger or multi-GPU machine if needed. For neural networks, Databricks recommends single-machine training when possible. A scikit-learn model runs on the driver, so extra workers add nothing. n_jobs uses only the driver's cores.
  2. Is it a classical algorithm that exists in pyspark.ml, on data too big for one node? Use Spark ML pipelines.
  3. Is it one model per store or region? Use groupBy().applyInPandas(). You get one task per group, so 12 regions keep only 12 tasks busy, however large the cluster is.
  4. Is it a deep model that fits on one GPU but trains on huge data? Use data parallelism with TorchDistributor or Ray Train.
  5. Do the weights not fit on one GPU? Use model parallelism, for example DeepSpeed.

For tuning, the answers are Optuna (with MlflowSparkStudy and shared MLflow storage) and Ray Tune. Open-source Hyperopt is no longer maintained and is not included in Databricks Runtime ML after 16.4 LTS ML, so it only appears as a wrong option. One shared-storage trap: with the default in-memory storage, each worker keeps its own study, so 40 workers run 40 separate searches that never learn from each other.

Going deeper: why a pipeline is the unit you tune and save

In Spark ML, an Estimator has fit() and returns a Model, and that Model is a Transformer with transform(). A Pipeline is itself an Estimator. Fitting it returns a PipelineModel in which every learning stage, such as StringIndexer, Imputer or StandardScaler, has become a fitted Transformer. That is why the whole pipeline goes into CrossValidator: preprocessing is refitted on each training fold, so statistics from the held-out fold never leak into training. It is also why the saved PipelineModel scores new data correctly: the learned indices and scaling travel with it. Preparing columns by hand outside the pipeline breaks both guarantees.

Hand-off 3: from a training run to a production version

Databricks recommends deploy code in most cases. The training code moves from development to staging to production and retrains in each environment, usually one Unity Catalog catalog per environment. Deploy models (promoting the trained artifact) is worth considering when training is very expensive or hard to reproduce, everything lives in one workspace, or there is no CI/CD.

Inside the production job, the training task registers a version and passes its exact URI to the validation task with task values. Asking the registry for "the latest version" can pick up another run's model if two runs overlap. A version that passes validation gets the Challenger alias. Deployment compares it with Champion and moves Champion only if the challenger is at least as good. Consumers load @Champion, so they follow the alias without code changes. Aliases replace the old stages.

The bundle side is tested too. Declarative Automation Bundles (formerly Databricks Asset Bundles) define experiments, registered_models and model_serving_endpoints as code. A mode: development target prefixes resources and pauses schedules, and production deployments should run as a service principal.

Hand-off 4: from a registered version to an endpoint

Model Deployment is only 12%, but the questions are precise. A custom pyfunc model loads heavy objects once in load_context, not in predict. It ships its files through artifacts, and it needs a signature to be registered in Unity Catalog. Send requests as dataframe_split: the records format does not guarantee column order.

RolloutFits whenBuilt from
CanaryHigh traffic, subtle regressions; raise the share in stepsTwo served entities, traffic_config (e.g. 90/10)
Blue-greenA breaking input change or an instant rollbackTwo endpoints; the client switches
Shadow / replayLow traffic, where a canary would prove nothingClient copy or replay job; traffic splits do not copy

You build these strategies yourself from served entities, traffic splits and endpoints. While an update is in progress, the old configuration keeps serving until the new one is ready.

Hand-off 5: from the endpoint to the monitor

The monitoring chain has a fixed shape. The inference table logs raw request and response JSON. A scheduled job unpacks it into typed columns and joins labels when they arrive. An inference profile attaches to that flat table and writes _profile_metrics and _drift_metrics tables. Finally, a Databricks SQL alert on those tables notifies someone or starts retraining. The profile's own notification settings report failed refreshes, not drift.

Know the drift details. Numeric columns get KS, Wasserstein and PSI. Categorical columns get chi-squared and distance measures, so a NULL ks_test on a category is expected. The documented PSI bands are below 0.1 (no significant change), below 0.2 (moderate) and 0.2 or above (significant). Baseline drift catches slow creep. Consecutive drift catches sudden steps. With no label column, accuracy stays NULL, but drift still works.

Going deeper: why a p-value alert fires every day

A p-value measures how detectable a shift is, not how large it is. With hundreds of millions of rows per window, almost any real-world difference is detectable, so a rule like ks_test.pvalue < 0.05 fires daily on shifts that do not matter. Testing many features makes it worse: 300 features at 0.05 give about 15 false alarms per refresh even when nothing changed. Alert on an effect size instead (the KS statistic, Wasserstein distance in the feature's own units, or PSI), with thresholds tied to business impact. When an MLP option offers "lower the p-value threshold" as the fix for noisy alerts, ask whether the stem has a large-sample problem.

DBX-MLP: the five hand-offs where most failures happen
1
Feature tables to training set
Time series feature table and point-in-time lookups, so labels never see the future
2
One machine to many
Climb the scaling ladder and stop at the first step that fits
3
Training run to production version
Deploy code, pass the exact model URI with task values, move the Champion alias
4
Registered version to endpoint
pyfunc loads in load_context; canary, blue-green or shadow rollout
5
Endpoint to monitor
Inference table, flat table, inference profile, then a SQL alert on drift
Follow a model from left to right: each card is one hand-off where one stage passes something to the next.

A study order

  1. Feature engineering: time series tables, point-in-time lookups, on-demand features, online stores.
  2. The scaling ladder and tuning: Spark ML pipelines, pandas function APIs, Optuna, Ray, TorchDistributor.
  3. The lifecycle: deploy code, task values, aliases, tests per stage, bundles.
  4. Monitoring: profile types, drift tables, alerts, custom metrics, slices.
  5. Deployment last: pyfunc packaging, querying, rollouts. Then mixed practice sets with the clock running.

Practise

Practise in the question bank: every question explains each option, right and wrong. Start with the domain you just studied, then take a timed mock exam.

Domains and weights as listed in the Databricks exam guide for the Machine Learning Professional certification. Databricks renames features often (Lakehouse Monitoring is now data profiling, online tables are now Online Feature Stores, and Feature Views are recommended over hand-written feature tables for new projects); check the current exam guide and documentation.