DBX-MLP Study Notes in Questions: Model Development

This set covers Model Development, one of the domains of the Databricks Machine Learning Professional (DBX-MLP) exam. It makes up about 44% of the exam. Try each question first, then open the answer to see why every option is right or wrong.

DBX-MLP Model Development: 5 practice questions

Databricks Machine Learning Professional (DBX-MLP) · Model Development · 5 questions with answers and explanations

🧭 Reading Model Development questions

Professional model development questions mix Spark ML, distributed training and custom pyfunc models at scale. Read the data shape and where training runs. Strong options use the API that fits the workload, batch or stream, single node or many; weaker ones apply an estimator or library where it cannot run.

Tips for this set

• Spark ML estimators fit on static DataFrames; a fitted PipelineModel can then score a stream.
• String labels need StringIndexer before a classifier, and IndexToString maps predictions back.
• Multi-node GPU errors such as NCCL failures usually point to networking or environment, not to the model code.

📊 Set profile

Exam: DBX-MLP · 59 questions · 120 minutes
This set: 3 applied, 2 advanced · 1 comparison, 1 concept check, 1 scenario, 1 trade-off, 1 troubleshooting
Topics: Spark ML, MLflow

Question 1 of 5

A fraud model is tuned with CrossValidator(numFolds=5). Each customer has many transactions, and cross-validation scores are far higher than the scores on later customers. The team suspects the folds. What change is the best fix?

AAdd a fold column such as abs(hash(customer_id)) % 5 and pass it with setFoldCol("fold")
BSet the seed of the CrossValidator so the same random folds are used for every parameter combination and run
CSwitch to TrainValidationSplit(trainRatio=0.8), which splits at random and so avoids sharing customers
DRaise numFolds to 20 so that each validation fold is smaller
Show answer

✅ Answer: A

Difficulty: advanced · Topic: Spark ML

Random folds leak groups
CrossValidator splits rows at random. If one customer's transactions are spread over the folds, the model sees that customer in training and is then validated on the same customer, memorizing customer-specific patterns. Scores are optimistic for new customers, which is the production situation.

foldCol
CrossValidator accepts foldCol, a column with user-specified fold numbers. Spark then skips random k-fold and uses your assignment. The column must be an integer in the range 0 to numFolds-1; out-of-range values raise an error. Assigning folds by a hash of the group key keeps all rows of a customer in the same fold.
• Make the fold assignment deterministic and independent of the label.
• Check fold sizes: very uneven groups give uneven folds.

Why each option works or fails
• ✓ A. (correct): foldCol with a customer-based fold number makes every customer appear in only one validation fold, so the score measures performance on unseen customers.
• ✗ B. Fixed seed: a seed makes the random folds repeatable but they stay random with respect to customers.
• ✗ C. TrainValidationSplit: it also splits rows at random into train and validation, so customers are still shared; it has no foldCol.
• ✗ D. More folds: the leakage comes from customers spread over folds, and smaller folds do not keep a customer together.

Question 2 of 5

Several models (churn, upsell, support risk) share one featurization: indexing four categorical columns, one-hot encoding them, and assembling a vector. The team wants one reusable, fitted artifact for the shared featurization and separate model stages per use case. Which design applies Spark ML correctly?

AFit the shared stages once with fit and call collect() on the result, then rebuild a Pipeline per model from the collected rows
BBuild a featurization Pipeline and use it as the first stage of each model Pipeline, since a Pipeline is itself an Estimator
CReplace the shared stages by a PipelineModel object in the stage list of each model pipeline after calling fit
DCreate three copies of the same stage objects and put one copy into each Pipeline, because a stage instance may be used in only one pipeline
Show answer

✅ Answer: B

Difficulty: applied · Topic: Spark ML

A Pipeline is an Estimator
A Pipeline has fit and returns a PipelineModel, so it is an Estimator and can be a stage in another Pipeline. The outer fit fits the nested pipeline (producing a nested PipelineModel) and passes its output to the later stages.

Why this helps
• The shared featurization is defined in one place and versioned once.
• Each model pipeline is Pipeline(stages=[featurization, classifier]), so the saved outer PipelineModel is a single artifact that scores raw columns.
• Tuning with CrossValidator still re-fits the nested stages per fold, as needed to avoid leakage.

When to separate
If many models must use the same fitted encodings (for instance, identical index values), fit the featurization once, save the PipelineModel, and reuse it as a transformer stage; accept that tuning then does not refit it.

Why each option works or fails
• ✗ A. Collect and rebuild: collect() moves the data to the driver, and rebuilding pipelines from rows throws away the fitted stage state and does not scale.
• ✓ B. (correct): Using the featurization Pipeline as the first stage of each model pipeline keeps one definition and gives a self-contained artifact per model.
• ✗ C. Fitted PipelineModel in the stage list: a fitted PipelineModel is a Transformer and can be placed as a stage, but then it is fixed to the training data of the first fit, and the option as written replaces rather than reuses the definition.
• ✗ D. Three stage copies: sharing a stage definition is exactly what nesting a pipeline provides; copying objects gives three diverging definitions, and the claim about single-use is not a rule.

Question 3 of 5

A daily job scores 15,000 rows with a scikit-learn model logged to MLflow. The feature table is 12 MB. The team's pipeline template always applies mlflow.pyfunc.spark_udf on a 12-worker cluster, and the job takes eight minutes, mostly start-up. What should an ML engineer recommend?

AConvert the sklearn model to Spark ML to remove the Python worker overhead, since JVM scoring is faster for 15,000 rows
BIncrease the cluster to 48 workers, because UDF scoring time falls in proportion to the number of executors for any data size
CSwitch to a streaming job with readStream, because streaming scoring has lower fixed cost than batch for small tables
DUse a small single-node cluster: read the table, load the model with mlflow.pyfunc.load_model, and call predict on pandas
Show answer

✅ Answer: D

Difficulty: applied · Topic: MLflow

Distribution has a fixed cost
Spark parallelism pays when data or computation is large. Starting executors, shipping the model to them and running tasks has overhead that does not shrink for small inputs. With 15,000 rows and a few megabytes of features, the scoring itself takes seconds on one machine.

Right-size the design
• Small, regular batches: single-node cluster, mlflow.pyfunc.load_model, toPandas() (or a read with pandas) and predict, then write the result back.
• Large tables or heavy models: spark_udf on a multi-worker cluster.
Keep one scoring contract (the logged model) while changing the execution to fit the volume.

Why each option works or fails
• ✗ A. Convert to Spark ML: it changes the model for no real gain at this volume, and the time was start-up, not Python overhead.
• ✗ B. 48 workers: more executors add start-up time and cost, and tiny partitions give no speedup, so scaling is not linear.
• ✗ C. Streaming: a stream adds checkpointing and trigger management and is not a cheaper way to score a table that changes once a day.
• ✓ D. (correct): At this size a single node loads the model once and scores quickly, with no cluster start-up for many workers; the logged model and its signature are reused unchanged.

Question 4 of 5

A feature age must become five bins. The business wants bins with roughly equal numbers of customers in the training data, learned once and reused unchanged when scoring later batches. Which stage fits?

ABinarizer applied five times with different thresholds, each producing one bin indicator
BQuantileDiscretizer(numBuckets=5), which learns quantile splits and returns a fixed Bucketizer
CStandardScaler, because z-scores split in the same way as quantiles
DBucketizer with splits computed again on each scoring batch, so the bins always follow the current customer population
Show answer

✅ Answer: B

Difficulty: applied · Topic: Spark ML

Estimator that learns, Transformer that applies
Bucketizer is a Transformer: you give it explicit splits and it maps values to bucket indices. QuantileDiscretizer is an Estimator: fit computes approximate quantiles of the input column and returns a Bucketizer model with those splits. The number of buckets can be smaller than requested if the input has too few distinct values.

Why it matters for scoring
The splits are learned once from the training data and stored in the fitted PipelineModel. New batches are binned with the same boundaries, so a customer of a given age always lands in the same bin, and the model's learned coefficients stay valid.
• Use Bucketizer when the business defines the boundaries.
• Use QuantileDiscretizer when equal-frequency bins should be learned from training data.

Why each option works or fails
• ✗ A. Repeated Binarizer: Binarizer thresholds a value to 0/1 and thresholds are fixed numbers you must choose, so it does not learn quantiles and gives overlapping indicators rather than one bin.
• ✓ B. (correct): QuantileDiscretizer learns equal-frequency boundaries from the training data and freezes them in a Bucketizer, which the pipeline applies unchanged to later batches.
• ✗ C. StandardScaler: it centers and scales values; it does not bin, and z-scores are not equal-frequency groups.
• ✗ D. Splits per scoring batch: boundaries recomputed on each batch change the meaning of every bin between batches and break the model's learned weights.

Question 5 of 5

A Ray cluster created with setup_ray_cluster is running. In the Spark UI the team sees one long-running Spark job whose number of tasks equals the number of Ray worker nodes, and it is not finishing. An engineer wants to kill it as "stuck". What is this job?

AA Spark ML training job that Ray submits for every trial, which should finish when the first trial completes
BThe Spark external shuffle service, which Ray starts so that its tasks can exchange objects through shuffle files between executors
CA broadcast of the Ray dependencies, which ends once every executor has copied the Python packages
DThe background Spark job that hosts the Ray cluster: one task per Ray worker node, with the head node on the driver
Show answer

✅ Answer: D

Difficulty: advanced

How Ray on Spark is built
ray.util.spark.setup_ray_cluster creates a Ray cluster *on* Apache Spark. Internally it starts a background Spark job. Each Spark task in that job starts a Ray worker node and keeps running, so the Ray worker lives as long as the task. The Ray head node is created on the Spark driver.
Consequences: Ray workers occupy Spark task slots for their whole lifetime; the number of Ray workers is set by min_worker_nodes and max_worker_nodes; calling shutdown_ray_cluster ends the job. By default the head node gets 0 CPUs and 0 GPUs, since the driver is shared.

Why each option works or fails
• ✗ A. Per-trial Spark ML job: Ray tasks run on the Ray workers, not as Spark ML jobs; this job exists once per Ray cluster, not per trial.
• ✗ B. Shuffle service: Ray has its own object store and does not use Spark shuffle files; the documented design is one task per Ray worker.
• ✗ C. Dependency broadcast: libraries are installed before the cluster starts and nothing is being broadcast; such a step would end quickly.
• ✓ D. The long-running job is the Ray cluster itself (correct): one Spark task per Ray worker node. Killing it takes the Ray cluster down, so the correct way to end it is shutting down the Ray cluster.

🎯 Key ideas from this set

• For grouped data, assign folds by group with foldCol so that no group is split across training and validation.
• A Pipeline is an Estimator, so a shared featurization Pipeline can be the first stage of many model pipelines.
• For small scoring batches use a single node with a loaded pyfunc model; use spark_udf only when volume warrants distribution.
• QuantileDiscretizer (an Estimator) learns equal-frequency splits and keeps them as a Bucketizer; Bucketizer alone applies fixed splits.
• Ray on Spark = a background Spark job with one task per Ray worker and the head on the driver.

🔗 Keep practicing on this blog

• All DBX-MLP practice sets
• Every exam on this blog

📚 Beyond this set

More DBX-MLP questions: https://hyunhp.github.io/cert-bank/#exam=DBX-MLP
Study guide: https://hyunhp.github.io/cert-bank/study.html#exam=DBX-MLP

These are original practice questions, not real exam questions, and cert-bank is not affiliated with any certification vendor.

Original practice questions written for this blog, not exam dumps.

Comments