Monitoring, Maintenance, and Security: 5 MLA-C01 Questions with Full Explanations
5 questions, one domain: Monitoring, Maintenance, and Security from the AWS Machine Learning Engineer – Associate (MLA-C01) exam. It carries about 24% of the exam score. Pick your answer, then check it against the explanation.
AWS Machine Learning Engineer – Associate (MLA-C01) · Monitoring, Maintenance, and Security · 5 questions with answers and explanations
🧭 How Monitoring, Maintenance, and Security is tested
Here the model is already in production and the stem reports a symptom: slow requests across services, accuracy that fades, idle endpoints, rising spend, or a permission that does not reach a job. First decide whether the need is to detect, prevent or explain a problem, then find the service that does exactly that. Tools that only analyze data later are common distractors.
Three things to look for
• When accuracy drops while the inputs look the same, the link between inputs and outcomes has changed. That calls for monitoring against ground-truth labels, not only input distribution checks.
• For a slow request that passes through several services, ask which signal can split the total time by service. Metrics and log lines alone rarely locate the slow hop.
• Separate alerting on spend from analyzing it. A requirement to be warned when a budget is forecast or passed needs a service that sends alerts, not a reporting tool.
Question 1 of 5
A SaaS company has 900 small XGBoost models, one per tenant, each on its own ml.m5.large real-time endpoint that is under 3% utilized. All models use the same container. The company wants to cut hosting cost and the number of endpoints to manage, and an occasional slower first request for a rarely used tenant is acceptable. Which option fits BEST?
Show answer
✅ Answer: C
Difficulty: applied · Topic: Amazon SageMaker, Amazon S3
Multi-model endpoints
A SageMaker multi-model endpoint (MME) serves many models that share one container image from a single endpoint.
• Model artifacts stay in an S3 prefix. The request names the model, for example TargetModel="tenant-417.tar.gz".
• On the first request for a model, the endpoint downloads and loads it into memory; later requests use the cached copy. Rarely used models are unloaded when memory is needed.
• Instances are shared, so utilization goes up and the number of endpoints drops from hundreds to a few.
When it fits
Many similar, individually small, low-traffic models with the same framework, where occasional cold-load latency is acceptable. Very latency-sensitive or very large models are better on dedicated capacity.
Why each option works or fails
• ✗ A. Smaller instances per tenant: cost falls somewhat, but the company still manages 900 endpoints with mostly idle capacity.
• ✗ B. Multi-container endpoint: it hosts a small number of different containers (up to 15), not hundreds of models that share one container.
• ✓ C. Multi-model endpoints (correct): hundreds of same-container models share a few instances, loaded from S3 on demand, so both cost and the number of endpoints fall sharply while the rare slow first request is accepted.
• ✗ D. Serverless endpoint per tenant: billing improves, but there are still 900 endpoints to manage, and serverless concurrency quotas are shared across the account.
Question 2 of 5
A team attaches an S3 read policy to an IAM group and also wants SageMaker jobs to read the same bucket. Which statement is correct?
Show answer
✅ Answer: C
Difficulty: advanced · Topic: AWS IAM
Policies versus roles
• A policy is a JSON document listing permissions, for example s3:GetObject on arn:aws:s3:::ml-data/*. It does nothing on its own until it is attached to an identity or a resource.
• Identities are users, groups and roles. A policy can be attached to any of them. A group is just a way to give the same policies to several users.
• A role is an identity without long-term credentials. A trusted principal, such as a user or an AWS service, assumes it and receives temporary credentials.
How SageMaker fits in
A SageMaker training job does not belong to any IAM group. It runs as the execution role whose trust policy allows sagemaker.amazonaws.com to assume it. To let jobs read the bucket, attach the S3 read policy (or an equivalent one) to that execution role as well.
Why each option works or fails
• ✗ A. Reversed definitions: it is the other way around; roles are assumable identities and policies are the permission documents.
• ✗ B. Policies only on roles: identity-based policies can be attached to users, groups and roles, as the team already did with the group.
• ✓ C. Policies grant permissions and SageMaker assumes a role (correct): permissions are defined in policies, and SageMaker jobs use them only when the policy is attached to the execution role the service assumes.
• ✗ D. Group policy applies to jobs: a group policy applies only to the IAM users in that group; a SageMaker job runs as its execution role and never inherits a group's permissions.
Question 3 of 5
A request intermittently takes 8 seconds. Metrics show the latency spike, logs show a timeout message, but the team can't tell which of 6 services caused it. Which signal should they use next?
Show answer
✅ Answer: C
Difficulty: advanced · Topic: AWS X-Ray, Amazon CloudWatch
Metrics, logs and traces
The three kinds of telemetry answer different questions:
• Metrics say *that* something is wrong: p99 latency rose to 8 s. They are aggregates over time, so they cannot follow one request.
• Logs give *details* of events inside one component: a line saying timeout calling inventory. But each service logs separately, so it is hard to see which of 6 hops actually used the time.
• Traces show *where* the time went. AWS X-Ray follows one request across all services with a shared trace ID, recording a segment per service and subsegments for downstream calls, each with start time and duration.
Example: a trace for an 8.1 s request shows API 8.1 s, orders 7.9 s, inventory 7.6 s, and the inventory database call 7.5 s. The slow hop is obvious.
X-Ray trace filters such as responsetime > 5 let you pull up just the slow requests.
Why each option works or fails
• ✗ A. CloudTrail data events: data events record API calls on AWS resources such as S3 objects or Lambda invokes; they show who called what, not how the request's time was spread across application services.
• ✗ B. 1-second metrics: higher resolution makes the spike sharper, but metrics are still aggregates per service and cannot follow one request across the call chain.
• ✓ C. X-Ray traces per service segment (correct): a trace follows a single request through all 6 services and shows how long each segment took, pinpointing the service responsible for the 8-second delay.
• ✗ D. Longer log retention: keeping logs longer does not add information; the existing logs already lack a view of the full request path.
Question 4 of 5
A demand model's accuracy drops slowly after a new competitor enters the market, though the input feature distributions look unchanged. What kind of drift is this, and how is it detected?
Show answer
✅ Answer: C
Difficulty: advanced · Topic: SageMaker Model Monitor
Data drift versus concept drift
A model learns a mapping from inputs X to a target y.
• Data drift (covariate shift): the distribution of X changes, for example customers suddenly come from a new region. Input distribution checks detect it without labels.
• Concept drift: the relationship between X and y changes while X can look the same. The same inputs now lead to different outcomes.
Example: a competitor enters the market. Store traffic, prices and seasonality features still look normal, but for the same inputs actual demand is lower because some customers go to the competitor. The model keeps predicting the old demand.
How to detect it
Because inputs look unchanged, input monitoring stays quiet. The problem only appears when predictions are compared with real outcomes (ground truth). SageMaker Model Monitor model quality monitoring merges captured predictions with ground truth labels and tracks metrics such as MAE or RMSE for regression, alerting when they degrade.
Why each option works or fails
• ✗ A. Data drift via input checks: data drift means the input distributions change, but the stem says they look unchanged, so input distribution checks would not flag anything.
• ✗ B. Latency drift: adding instances addresses slow responses, not a gradual loss of accuracy caused by a changed market.
• ✓ C. Concept drift detected with model quality monitoring (correct): the input distributions are stable but accuracy falls, which means the input-to-target relationship changed; only comparing predictions with ground truth labels can reveal that.
• ✗ D. Hardware drift: instance type affects speed and cost; the model computes the same predictions on any hardware, so changing it cannot restore accuracy.
Question 5 of 5
A team wants an email when forecast monthly spend will exceed $10,000, and also wants a Slack alert when actual spend passes 80% of that budget. Which service fits BEST?
Show answer
✅ Answer: B
Difficulty: advanced · Topic: AWS Budgets
AWS Budgets in one paragraph
AWS Budgets lets you set a spending (or usage, RI or Savings Plans) target and attach alerts that fire on thresholds.
• Each alert can compare against actual spend (what has already accrued this month) or forecasted spend (what AWS predicts the month will end at).
• Thresholds can be absolute amounts or percentages of the budget, for example "forecast > 100%" and "actual > 80%".
• Notifications go to email addresses and to an Amazon SNS topic. From SNS, or directly from Budgets, Amazon Q Developer in chat applications (formerly AWS Chatbot) can post the alert to a Slack channel.
Example
Budget monthly-total = $10,000. Alert 1: forecasted > 100% to finance@example.com. Alert 2: actual > 80% to an SNS topic subscribed by the chat integration, so the Slack channel is told soon after accrued spend passes $8,000 (Budgets refreshes cost data several times a day, not in real time). Budget actions can even apply an IAM policy or SCP automatically if needed.
Why each option works or fails
• ✗ A. CUR to S3: the Cost and Usage Report is a raw billing data file for analysis; it has no thresholds and sends no notifications.
• ✓ B. AWS Budgets forecast and actual alerts (correct): one budget holds both a forecast-based and an actual-based threshold and delivers them by email and SNS, which reaches Slack through the chat integration.
• ✗ C. Cost Explorer saved reports: Cost Explorer visualizes and forecasts cost when someone opens it, but a saved report does not push an alert when a threshold is crossed.
• ✗ D. Trusted Advisor cost checks: these flag optimization opportunities like idle load balancers; they do not track spend against a monthly target.
🎯 Key ideas from this set
• Many small models on the same container with low traffic → multi-model endpoint; multi-container endpoints are for a few different containers.
• Policies hold permissions and roles are what services act as, so a job gets S3 access only through the policy on its execution role.
• Metrics show that something is wrong, logs show what happened in one place, and traces show where the time went across services.
• Same inputs, different outcomes is concept drift, caught by model quality monitoring against ground truth, not by input distribution checks.
• Proactive alerts on forecast or actual spend are what AWS Budgets is for; Cost Explorer and the CUR analyze, they do not alert.
📊 About this set
Exam: MLA-C01 · 65 questions · 130 minutes · passing score 720
This set: 4 advanced, 1 applied · 3 concept check, 1 comparison, 1 scenario
Topics: Amazon SageMaker, Amazon S3, AWS IAM, AWS X-Ray, Amazon CloudWatch, SageMaker Model Monitor, AWS Budgets
🔗 Keep practicing on this blog
• 5 Questions to Test Your ML Model Development Knowledge (MLA-C01)
• Can You Answer These 5 MLA-C01 Questions on Deployment and Orchestration?
• All MLA-C01 practice sets
• Every exam on this blog
📚 Keep studying MLA-C01
More MLA-C01 questions: https://hyunhp.github.io/cert-bank/#exam=MLA-C01
Study guide: https://hyunhp.github.io/cert-bank/study.html#exam=MLA-C01
These are original practice questions, not real exam questions, and cert-bank is not affiliated with any certification vendor.
Original practice questions written for this blog, not exam dumps.
Comments
Post a Comment